Method and apparatus for generating artificial intelligence training data

Abstract

Provided are a method by which a first device performs wireless communication and an apparatus supporting same. The method may comprise the steps of: acquiring a first image comprising a first object, on the basis of at least one sensor; receiving a message comprising information related to the first object; and generating training data on the basis of a background image, a cropped image of the first object acquired from the first image, and the information related to the first object.

Description

Method and device for generating artificial intelligence training data

[0001] The present disclosure relates to a wireless communication system.

[0002] 5G NR is a successor technology to LTE (long term evolution) and is a new clean-slate type mobile communication system with characteristics such as high performance, low latency, and high availability. 5G NR can utilize all available spectrum resources, ranging from low frequency bands below 1 GHz to mid-frequency bands from 1 GHz to 10 GHz, and high frequency (millimeter wave) bands above 24 GHz.

[0003] The 6G (wireless communication) system aims for (i) very high data rates per device, (ii) a very large number of connected devices, (iii) global connectivity, (iv) very low latency, (v) reduced energy consumption of battery-free IoT (internet of things) devices, (vi) ultra-reliable connectivity, and (vii) connected intelligence with machine learning capabilities. The vision of the 6G system can be in four aspects: intelligent connectivity, deep connectivity, holographic connectivity, and ubiquitous connectivity, and the 6G system can satisfy requirements such as those shown in Table 1 below. For example, Table 1 may represent an example of the requirements for a 6G system.

[0004] Per device peak data rate 1 Tbps E2E latency 1 ms Maximum spectral efficiency 100 bps / Hz Mobility support up to 1000 km / hr Satellite integration Fully AI Fully Autonomous vehicle Fully XR Fully Haptic communication Fully

[0005] In one embodiment, a method is provided in which a first device performs wireless communication. The method may include: acquiring a first image including a first object based on at least one sensor; receiving a message including information related to the first object; and generating training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0006] In one embodiment, a first device configured to perform wireless communication is provided. The first device may include at least one transceiver; at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, the instructions may cause the first device, based on execution by the at least one processor: to acquire a first image including a first object based on at least one sensor; to receive a message including information related to the first object; and to generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0007] In one embodiment, a processing device configured to control a first device is provided. The processing device comprises at least one processor; and at least one memory connected to the at least one processor and storing instructions, wherein the instructions, based on execution by the at least one processor, cause the first device to: acquire a first image including a first object based on at least one sensor; receive a message including information related to the first object; and generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0008] In one embodiment, a non-transient computer-readable storage medium is provided for recording instructions. When the instructions are executed, the first device may: acquire a first image including a first object based on at least one sensor; receive a message including information related to the first object; and generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0009] FIG. 1 shows a communication structure that can be provided in a 6G system according to one embodiment of the present disclosure.

[0010] FIG. 2 shows an electromagnetic spectrum according to one embodiment of the present disclosure.

[0011] FIG. 3 illustrates a method for generating AI training data using V2X infrastructure according to one embodiment of the present disclosure.

[0012] FIG. 4 illustrates a bounding box-based object detection method according to one embodiment of the present disclosure.

[0013] FIG. 5 illustrates a pixel-unit segmentation-based object detection method according to one embodiment of the present disclosure.

[0014] FIG. 6 illustrates an example of secondary inference according to one embodiment of the present disclosure.

[0015] FIG. 7 shows a background image that does not include a road user for generating training data according to one embodiment of the present disclosure.

[0016] FIG. 8 shows an object cropping image for generating training data according to one embodiment of the present disclosure.

[0017] FIG. 9 shows a result image obtained by synthesizing a background image and an object cropping image according to one embodiment of the present disclosure.

[0018] FIG. 10 shows a modular system for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure.

[0019] FIG. 11 shows an object detection system for generating artificial intelligence learning data based on V2X infrastructure according to one embodiment of the present disclosure.

[0020] FIG. 12 shows a software block for generating artificial intelligence learning data based on V2X infrastructure according to one embodiment of the present disclosure.

[0021] FIG. 13 illustrates a method for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure.

[0022] FIG. 14 illustrates a method in which a first device performs wireless communication according to one embodiment of the present disclosure.

[0023] FIG. 15 illustrates a method in which a second device performs wireless communication according to one embodiment of the present disclosure.

[0024] FIG. 16 shows a communication system (1) according to one embodiment of the present disclosure.

[0025] FIG. 17 shows a wireless device according to one embodiment of the present disclosure.

[0026] FIG. 18 shows a signal processing circuit for a transmission signal according to one embodiment of the present disclosure.

[0027] FIG. 19 shows a wireless device according to one embodiment of the present disclosure.

[0028] FIG. 20 shows a portable device according to one embodiment of the present disclosure.

[0029] FIG. 21 shows a vehicle or an autonomous vehicle according to one embodiment of the present disclosure.

[0030] In this specification, "A or B" may mean "only A," "only B," or "both A and B." Alternatively, in this specification, "A or B" may be interpreted as "A and / or B." For example, in this specification, "A, B or C" may mean "only A," "only B," "only C," or "any combination of A, B and C."

[0031] A slash ( / ) or a comma used in this specification may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B or C."

[0032] In this specification, "at least one of A and B" may mean "only A," "only B," or "both A and B." Additionally, in this specification, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as synonymous with "at least one of A and B."

[0033] Additionally, in this specification, "at least one of A, B and C" may mean "only A," "only B," "only C," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0034] Additionally, parentheses used in this specification may mean "for example." Specifically, when indicated as "control information (PDCCH)," "PDCCH" may be proposed as an example of "control information." In other words, "control information" in this specification is not limited to "PDCCH," and "PDCCH" may be proposed as an example of "control information." Furthermore, even when indicated as "control information (i.e., PDCCH)," "PDCCH" may be proposed as an example of "control information."

[0035] In the following explanation, 'when, if, in case of' can be replaced with 'based on'.

[0036] Technical features described individually within a single drawing in this specification may be implemented individually or simultaneously.

[0037] In this specification, a higher layer parameter may be a parameter that is set for the terminal, pre-set, or pre-defined. For example, a base station or a network may transmit the higher layer parameter to the terminal. For example, the higher layer parameter may be transmitted via radio resource control (RRC) signaling or medium access control (MAC) signaling.

[0038] In this specification, "set or defined" may be interpreted as being set or pre-configured to the device through pre-defined signaling (e.g., SIB, MAC, RRC) from a base station or network. In this specification, "set or defined" may be interpreted as being pre-configured to the device.

[0039] The technology proposed in this specification can be used in various wireless communication systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access). CDMA can be implemented with wireless technologies such as UTRA (universal terrestrial radio access) or CDMA2000. TDMA can be implemented with wireless technologies such as GSM (global system for mobile communications) / GPRS (general packet radio service) / EDGE (enhanced data rates for GSM evolution). OFDMA can be implemented with wireless technologies such as IEEE (institute of electrical and electronics engineers) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802-20, E-UTRA (evolved UTRA), LTE (long term evolution), and 5G NR.

[0040] The technology proposed in this specification can be implemented as 6G wireless technology and can be applied to various 6G systems. For example, 6G systems may have key factors such as eMBB (enhanced mobile broadband), URLLC (ultra-reliable low latency communications), mMTC (massive machine-type communication), AI (artificial intelligence) integrated communication, tactile internet, high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion, and enhanced data security.

[0041] FIG. 1 illustrates a communication structure that can be provided in a 6G system according to one embodiment of the present disclosure. The embodiment of FIG. 1 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0042] New network characteristics in 6G may be as follows.

[0043] - Satellite Integrated Network

[0044] - Connected Intelligence: Unlike previous generations of wireless communication systems, 6G is innovative and will update wireless evolution from "connected things" to "connected intelligence." AI can be applied at each stage of the communication process (or at each step of the signal processing described below).

[0045] - Seamless integration of wireless information and energy transfer

[0046] - Ubiquitous Super 3D Connectivity: Connectivity to the network and core network functions of drones and very low Earth orbit satellites will create Super 3D connectivity in 6G ubiquitous.

[0047] Some general requirements regarding the new network characteristics of 6G mentioned above may be as follows.

[0048] - Small cell networks

[0049] - Ultra-dense heterogeneous network

[0050] - High-capacity backhaul

[0051] - Radar technology integrated with mobile technology: High-precision localization (or location-based services) through communication is one of the functions of 6G wireless communication systems. Therefore, radar systems will be integrated with 6G networks.

[0052] - Softwarization and virtualization

[0053] The core implementation technologies of the 6G system are described below.

[0054] - Artificial Intelligence: Introducing AI into communications can streamline and enhance real-time data transmission. AI can determine how complex target tasks are performed using numerous analyses. In other words, AI can increase efficiency and reduce processing latency. Time-consuming tasks such as handover, network selection, and resource scheduling can be performed instantly using AI. AI can also play a significant role in M2M, machine-to-human, and human-to-machine communication. Furthermore, AI can enable rapid communication in Brain-Computer Interfaces (BCI). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radios, self-sustaining wireless networks, and machine learning.

[0055] - THz communication: Data transmission rates can be increased by increasing bandwidth. This can be achieved by using sub-THz communication with wide bandwidth and applying advanced large-scale MIMO technology. THz waves, also known as sub-millimeter radiation, generally represent a frequency band between 0.1 THz and 10 THz with corresponding wavelengths in the range of 0.03 mm to 3 mm. The 100 GHz to 300 GHz band range (Sub-THz band) is considered the main part of the THz band for cellular communication. Adding the Sub-THz band to the mmWave band increases 6G cellular communication capacity. Among the defined THz bands, 300 GHz to 3 THz is located in the far-infrared (IR) frequency band. The 300 GHz to 3 THz band is part of the broadband but lies at the boundary of the broadband and immediately following the RF band. Therefore, this 300 GHz to 3 THz band exhibits similarity to RF. FIG. 2 shows an electromagnetic spectrum according to one embodiment of the present disclosure. The embodiment of FIG. 2 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of said embodiments may be omitted. Key characteristics of THz communication include (i) widely available bandwidth to support very high data transmission rates, and (ii) high path loss occurring at high frequencies (highly directional antennas are indispensable). The narrow beam width generated by highly directional antennas reduces interference. The small wavelength of THz signals allows a much larger number of antenna elements to be integrated into devices and BSs operating in this band. This enables the use of advanced adaptive array techniques that can overcome range limitations.

[0056] - Large-scale MIMO technology

[0057] - Hologram beamforming (HBF)

[0058] - Optical wireless technology

[0059] - Free Space Optical Transmission Backhaul Network (FSO backhaul network)

[0060] - Quantum communication

[0061] - Cell-free communication

[0062] - Integration of wireless information and power transmission

[0063] - Integration of wireless communication and sensing

[0064] - Integrated access and backhaul network

[0065] - Big data analysis

[0066] - Reconfigurable intelligent metasurface

[0067] - Metaverse

[0068] - blockchain

[0069] - Unmanned Aerial Vehicle (UAV): UAVs or drones will be a critical element in 6G wireless communication. In most cases, high-speed data wireless connectivity can be provided using UAV technology. Base station (BS) entities can be installed on UAVs to provide cellular connectivity. UAVs can possess specific features not found in fixed BS infrastructure, such as easy deployment, robust line-of-sight links, and controlled degrees of freedom for mobility. During emergencies, such as natural disasters, the deployment of ground communication infrastructure is not economically feasible, and sometimes services cannot be provided in volatile environments. UAVs can easily handle these situations. UAVs will become a new paradigm in the field of wireless communication. This technology facilitates the three fundamental requirements of wireless networks: eMBB, URLLC, and mMTC. UAVs can also support various purposes, such as enhancing network connectivity, fire detection, disaster emergency services, security and surveillance, pollution monitoring, parking monitoring, and accident monitoring. Therefore, UAV technology is recognized as one of the most critical technologies for 6G communication.

[0070] - Advanced Air Mobility (AAM): AAM is a higher-level concept than Urban Air Mobility (UAM), which refers to air transportation available in urban areas; it can refer to a means of transportation that includes movement between regional hubs as well as within the city center.

[0071] - Autonomous Driving (Self-Driving): V2X (Vehicle to Everything), a core element in building autonomous driving infrastructure, refers to technologies that enable vehicles to communicate and share with various elements on the road for autonomous driving, such as wireless communication between vehicles (Vehicle to Vehicle, V2V) and between vehicles and infrastructure (Vehicle to Infrastructure, V2I). Fast transmission speeds and low-latency technologies are essential to maximize autonomous driving performance and ensure high safety. Furthermore, future autonomous driving may go beyond merely delivering warning or guidance messages to the driver to actively intervene in vehicle operation and directly control the vehicle in dangerous situations. Since the amount of information to be transmitted and received may become massive for this purpose, it is expected that 6G will be able to maximize autonomous driving through faster transmission speeds and lower latency compared to 5G.

[0072] Meanwhile, regarding V2X communication, prior to NR, radio access technology (RAT) mainly discussed methods for providing safety services based on V2X messages such as Basic Safety Message (BSM), Cooperative Awareness Message (CAM), and Decentralized Environmental Notification Message (DENM). For example, V2X messages may include location information, dynamic information, attribute information, etc. For example, a terminal may transmit a CAM of the periodic message type and / or a DENM of the event-triggered message type to another terminal.

[0073] For example, the CAM may include dynamic state information of the vehicle, such as direction and speed; static data of the vehicle, such as dimensions; and basic vehicle information, such as external lighting conditions or route history. For example, a terminal may broadcast the CAM, and the latency of the CAM may be less than 100ms. For example, in the event of an unexpected situation such as a vehicle breakdown or accident, the terminal may generate a DENM and transmit it to other terminals. For example, all vehicles within the transmission range of the terminal may receive the CAM and / or DENM. In this case, the DENM may have a higher priority than the CAM.

[0074] In addition, regarding V2X communication, various V2X scenarios are presented in NR. For example, various V2X scenarios may include vehicle platooning, advanced driving, extended sensors, remote driving, etc.

[0075] For example, based on vehicle platooning, vehicles can dynamically form groups and move together. For example, to perform platoon operations based on vehicle platooning, vehicles belonging to said group can receive periodic data from the lead vehicle. For example, vehicles belonging to said group can use said periodic data to reduce or increase the distance between vehicles.

[0076] For example, based on enhanced driving, vehicles can be semi-automated or fully automated. For example, each vehicle can adjust trajectories or maneuvers based on data acquired from local sensors of nearby vehicles and / or nearby logical entities. Additionally, for example, each vehicle can mutually share driving intentions with nearby vehicles.

[0077] For example, based on extended sensors, raw data or processed data or live video data acquired through local sensors can be exchanged between vehicles, logical entities, pedestrian terminals and / or V2X application servers. Thus, for example, a vehicle can perceive an environment that is enhanced compared to the environment it can detect using its own sensors.

[0078] For example, based on remote driving, a remote driver or V2X application can operate or control a remote vehicle for a person unable to drive or for a remote vehicle located in a dangerous environment. For example, in cases where the route is predictable, such as in public transportation, cloud computing-based driving can be used for the operation or control of the remote vehicle. Additionally, access to a cloud-based back-end service platform, for example, can be considered for remote driving.

[0079] Meanwhile, methods to specify service requirements for various V2X scenarios, such as vehicle platooning, enhanced driving, extended sensors, and remote driving, are being discussed in NR-based V2X communication.

[0080] Meanwhile, object detection systems utilizing camera sensors are a widely used technology in the field of artificial intelligence (AI), and significant research is currently being conducted in this area. However, there are many variables that can affect detection performance, such as differences in regional road infrastructure, vehicle shapes, camera angles of view, weather, seasonal variations, and road blockages. Overcoming these challenges requires a large amount of data trained on specific regions and environments. Regarding the collection of training data, however, while there are issues concerning personal data protection, the primary problem is that the annotation process, such as labeling the collected data, entails a significant amount of time and effort.

[0081] Much research is being conducted to automate annotation and training, and foundation models trained on large datasets, such as SAM (Segment Anything Model) and DINO (Discriminator Interpolation), are being introduced. However, while automated annotation suitable for ITS road environments is well-suited for utilizing V2X infrastructure, publicly available overseas models often require additional training due to differences in road infrastructure and geometry compared to domestic environments. Therefore, there is a need for technology and research capable of automatically performing annotation and training within the V2X infrastructure environment.

[0082] In this disclosure, a method for performing an automated learning function on data required for artificial intelligence learning in an infrastructure that detects V2X road users through camera sensors such as CCTV, and a device supporting the same are proposed.

[0083] For example, when detecting objects in video footage acquired by camera sensors using infrastructure built on roads (e.g., CCTV, RSU, etc.), object detection information including the location and area of ​​the object can be utilized. When matching these acquired objects with objects in V2X messages, data labeling can be performed by utilizing type information of V2X road users. However, road users not connected to V2X cannot be matched with objects in V2X messages and must therefore be excluded from the acquired video footage. For instance, to address this, background images excluding road users can be captured or created and stored. Then, video footage for training can be automatically generated by cropping the object detection areas for V2X-connected users through image processing and compositing them with the background image. This allows for the automation of data annotation processes, such as data labeling.

[0084] In addition, for example, since the matching between an object detected by a camera sensor and an object recognized by a V2X message may be incorrectly matched due to confusion of objects depending on the accuracy of GNSS (global navigation satellite system) positioning data, the present disclosure additionally proposes several methods to verify object matching in addition to installing a separate positioning sensor.

[0085] In addition, we propose a method and device that utilize V2X cognitive information for training automation when available, so that even if a camera sensor fails to detect an object due to factors such as season, weather, obstruction, object size, distance, or rapid movement, a large amount of training data can be secured.

[0086] For example, when utilizing existing road infrastructure such as CCTVs, training data can be automatically generated 24 hours a day, 365 days a year, depending on the operating time of the CCTVs. Since accurate data must be secured at this time, it is important to accurately match object information detected by camera sensors with V2X object information. To this end, the present disclosure additionally proposes methods for verifying object matching, thereby enabling the securing of a large amount of accurate training data. Furthermore, the present disclosure proposes a method and apparatus that allow the V2X infrastructure to compensate for a decrease in the object detection rate of the sensor even if object detection fails by the sensor, by utilizing object recognition information received via V2X (e.g., V2X message) as training data.

[0087] FIG. 3 illustrates a method for generating AI training data using V2X infrastructure according to one embodiment of the present disclosure. The embodiment of FIG. 3 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0088] To explain the application method of the present disclosure, a method for matching and verifying an object on a vision sensor and a V2X message, and a method for automating artificial intelligence learning from the matched object information, are described as follows through examples of each step illustrated in FIG. 3.

[0089] For example, the object detection (300) of FIG. 3 can be detected by classifying objects on an image or by image pixels. For example, it is common to use either a detection technique that detects objects based on bounding boxes (see FIG. 4) or a segmentation technique that detects objects precisely by pixels (see FIG. 5).

[0090] FIG. 4 illustrates a bounding box-based object detection method according to one embodiment of the present disclosure. The embodiment of FIG. 4 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of the embodiments may be omitted.

[0091] FIG. 5 illustrates a pixel-based segmentation object detection method according to one embodiment of the present disclosure. The embodiment of FIG. 5 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0092] For example, in the case of the detection technique described above, the detection information may include a bounding box, and in the case of the segmentation technique described above, information including a segmentation map or a segment map may be included in the detection information (see 311 in FIG. 3). For example, in addition to the location of the object, the detection information may include the class, which is the classification of the object, confidence or score, which indicates the confidence of the classification, and the total number of objects included in the image.

[0093] For example, matching object information detected by a camera sensor with V2X object information is generally possible through matching GNSS positioning information (see 330 in FIG. 3). However, since positioning information may not be accurate in GNSS blind spots, to obtain information to assist and verify this, secondary inference to obtain detailed feature information such as vendor and vehicle type (see 320 in FIG. 3), generation of a tree (e.g., n-ary tree) or graph to verify the relative arrangement of objects located in the lane (see 321 in FIG. 3), or acquisition of protection level or covariance information representing the GNSS position probability of each object (see 322 in FIG. 3) may be performed. Additionally, for example, a positioning sensor may be added or other methods may be combined to be used as a means to assist GNSS positioning matching. Alternatively, for example, the matching verification procedure may be omitted, in which case the procedures 320, 321, and 322 of FIG. 3 may be unnecessary (thus, the procedures are indicated in parentheses).

[0094] FIG. 6 illustrates an example of secondary inference according to one embodiment of the present disclosure. The embodiment of FIG. 6 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of the embodiments may be omitted.

[0095] Referring to FIG. 6, the vendor and type of the vehicle can be detected through secondary inference after object detection, and other features such as the color of the vehicle can also be detected in addition to the type of vehicle. For example, through additional inference information, it can be used as auxiliary verification information to check whether an object in a V2X message matches a specific object on a camera, even if the GNSS positioning information is inaccurate. For example, in FIG. 6, the positioning information of the left vehicle (610) and the right vehicle (620) may be inaccurate and swapped, but if there is vehicle vendor or color information in the V2X message, this can be verified to increase the confidence score or score, indicate that the information is incorrect, directly correct the swapped information, or perform matching based on all available information.

[0096] Alternatively, for example, it is possible to obtain detailed features through second-order inference, but since this can affect system load or performance, the relative positions of objects based on lane arrangement can be simply verified. For example, in a road environment with n lanes, the detected objects can be represented as nodes such as an n-ary tree, graph, or doubly linked list to simply verify their relative positions relative to the lanes (see 321 in Fig. 3). Alternatively, for example, even in the case of an intersection where lanes are unclear, the relative positions can be compared by dividing the area in a manner generally provided by road and lane topology (RLT) services.

[0097] Alternatively, for example, since the matching method is based on GNSS positioning, it may be reasonable to verify the matching by obtaining detailed GNSS positioning information. To this end, logic can be added to verify whether the positioning information of the objects is correct by utilizing data such as positioning probabilities like GNSS HPL (horizontal protection level), covariance, confidence, and score (see 322 in Fig. 3).

[0098] Alternatively, for example, if sensor object detection information is available, it can be mapped one-to-one with V2X perceived object information (see 323 in FIG. 3), and a verification procedure can be performed as needed (see 324 in FIG. 3). However, for example, if sensor object detection information is missing due to various reasons such as seasonal vegetation, weather, or occlusion, or if it is determined that only V2X perceived object information exists and there is no corresponding sensor information, object information can be integrated so that learning automation can be performed using only V2X perceived information (see 330 in FIG. 3). For example, if sensor object detection information is information A and V2X perceived object information is information B, it can be viewed as an integration process in which all of information B is retrieved and only the joined information of A is retrieved (e.g., A RIGHT OUTER JOIN B).

[0099] For example, once integration is completed based on the matching of objects on the vision sensor and objects on V2X and V2X recognition information, only the objects connected to V2X among the detected objects can be included in deep learning or machine learning training. For example, all objects not connected to V2X can also be included in training, but in this case, it may be difficult to perform data annotation using V2X information. Therefore, a procedure for cropping and synthesizing V2X-linked objects on an image can be performed. To this end, for example, a background image that does not include a road user can be obtained in advance (see Fig. 7) and synthesized with the cropped image (see Fig. 8) (see 340 in Fig. 3).

[0100] FIG. 7 shows a background image not including road users for generating training data according to one embodiment of the present disclosure. The embodiment of FIG. 7 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of said embodiments may be omitted.

[0101] Referring to Fig. 7, as described above, in order to exclude objects not connected to V2X from the training data, training data can be generated by utilizing a background image without objects.

[0102] FIG. 8 shows an object cropping image for generating training data according to one embodiment of the present disclosure. The embodiment of FIG. 8 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of said embodiments may be omitted.

[0103] Referring to Fig. 8, as described above, a cropping image of each V2X connected vehicle to be composited onto a background image can be obtained, and V2X additional information can be matched to each object.

[0104] FIG. 9 shows a result image obtained by compositing a background image and an object cropping image according to one embodiment of the present disclosure. The embodiment of FIG. 9 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of the embodiments may be omitted.

[0105] Referring to FIG. 9, training data can be generated by synthesizing the background image of FIG. 7 described above and the object cropping image of FIG. 8 described above. Through this, training data excluding objects not connected to V2X can be generated and secured.

[0106] For example, a V2X-linked object cropping image combined with a background image can be used as image data for automatic learning, and the metadata of each cropping image can utilize object class or subclass information included in object detection information (see 311, 320 in FIG. 3) or utilize information included in V2X messages (see 341 in FIG. 3). For example, as an example of utilizing information included in V2X messages, "vehicleSubClassType" can be utilized as object classification information in the CPM (collective perception message) (or sensor sharing message) of the ETSI ITS standard or SAE standard, or the current external live state of the CAM (cooperative awareness message) (or awareness message) can be merged to be learned as very detailed information such as "heavy equipment truck with vehicle lights turned on."

[0107] For example, synthetic images and metadata of each object can be integrated into training data and transmitted to the Cloud or Edge (see 342 in FIG. 3), and after the transmitted training data is collected (see 350 in FIG. 3), transfer learning or retraining can be performed in the Cloud or Edge (see 351 in FIG. 3).

[0108] FIG. 10 illustrates a modular system for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure. The embodiment of FIG. 10 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0109] Referring to FIG. 10, a modular system for generating artificial intelligence training data based on V2X infrastructure may include multiple cases. For example, multiple cases may be interconnected via an Ethernet network. For example, at least one PC (internal PC) included in case 1 may operate as a master, and this master may perform handover or fusion. For example, if a fault occurs in the master PC, a client PC included in another case may take over the role of the master PC. For example, among multiple client PCs excluding the master PC, a priority may be set for the client PC that can take over the role of the master PC. For example, through the division of roles between the master PC and the client PC, stable handover and fusion functions can be performed, and service continuity can be guaranteed even if the master PC fails.

[0110] FIG. 11 illustrates an object detection system for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure. The embodiment of FIG. 11 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0111] Referring to FIG. 11, an object detection system for generating artificial intelligence training data based on V2X infrastructure may include a smart RSU, at least one camera, a PoE (power over Ethernet) injector, a power outlet, a wired router, a device equipped with object detection, fusion, and handover modules, and a remote power control device (iBoot-G2+). For example, the above configuration may be based on Sensor Fusion 2.0 Base. For example, if at least one sensor is added to the above configuration, it may be based on Sensor Fusion 2.0 Advanced. For example, the power outlet (220V AC) may be connected to the PoE injector, the wired router (or a wired router including PoE functionality), and the remote power control device to supply power. For example, the smart RSU may be connected to the PoE injector via an Ethernet+PoE cable. For example, at least one camera can be connected to a wired router via an Ethernet+PoE cable. For example, if at least one sensor is added, at least one sensor can be connected to a wired router via an Ethernet+PoE cable. For example, a device equipped with object detection, fusion, and handover modules can be connected to a wired router via an Ethernet cable. For example, a remote power control device can be connected to a wired router via an Ethernet cable.

[0112] FIG. 12 illustrates a software block for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure. The embodiment of FIG. 12 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0113] Referring to FIG. 12, video or image data received from at least one camera can be decoded through a decoder, and then an object included in the video or image can be detected through an object detection module. For example, the detected object can be managed continuously through a handover module. For example, a matching module can perform matching between the object detected by the sensor and the information in the V2X message. For example, data input from an additional sensor can undergo a pre-processing process and be fused with the information obtained from the matching module. For example, the aforementioned data / information can be integrated and processed in a fusion module. For example, the data / information integrated and processed in the fusion module can be converted into a transmittable form through a packetizing module. For example, the packetized data can be transmitted to an external network via a smart RSU. For example, it can be delivered to a server via the internet. For example, the server can perform artificial intelligence learning based on collected data. For example, the server can provide the data acquired based on artificial intelligence learning to the user.

[0114] FIG. 13 illustrates a method for generating artificial intelligence training data based on V2X infrastructure according to one embodiment of the present disclosure. The embodiment of FIG. 13 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0115] Referring to FIG. 13, in step S1310, if a sensor included in the V2X infrastructure (e.g., camera, LiDAR, radar, etc.) detects an object (e.g., V2X connected object), object detection information can be transmitted to an object detector (see 310 in FIG. 3). In step S1315, an object detector (or RSU (Road Side Unit)) included in the V2X infrastructure can transmit detection information, such as a zone containing the object (e.g., bounding box), to a V2X server (e.g., Edge, Cloud) (see 311 in FIG. 3). In step S1320a, an object detector (or RSU) included in the V2X infrastructure can receive V2X object recognition information (e.g., V2X message) from a V2X server (e.g., Edge, Cloud) (see 312, 313 in FIG. 3). In step S1320b, a V2X server (e.g., Edge, Cloud) can transmit V2X object recognition information to the UE (optional). In step S1325, an object detector (or RSU) included in the V2X infrastructure can perform secondary inference, etc., on the detected object (see 320, 321, 322 in FIG. 3) (optional). For example, the object detector (or RSU) can map information related to the object detected based on the sensor to the object included in the V2X message (see 323 in FIG. 3) (optional). For example, the object detector (or RSU) can perform mapping of information related to the object detected by the sensor to the object included in the V2X message whenever the object is detected by the sensor (e.g., N times). For example, an object detector (or RSU) can perform mapping verification of information related to an object detected by a sensor and an object included in a V2X message (see 324 in FIG. 3) (optional). For example, mapping verification of information related to an object detected by a sensor and an object included in a V2X message can be performed whenever an object is detected by a sensor (e.g., N times).In step S1330, if the object detector (or RSU) has performed N mapping and / or mapping verifications, the object detector (or RSU) can integrate N V2X object detection / mapping information (see 330 in FIG. 3). In step S1335, the object detector (or RSU) can crop only the object from an image containing the object and can composite the cropped image of the object with a background image (e.g., a background image containing no object) (see 340 in FIG. 3). In step S1340, the object detector (or RSU) can annotate information about each object to each object (see 341 in FIG. 3). For example, the object detector (or RSU) can annotate information about the object to the composite image of the cropped image of the object and the background image obtained in the above-described step S1335. For example, the operation of annotating information about an object on a composite image of a cropped image of an object and a background image can be performed automatically. In step S1345, the object detector (or RSU) can transmit data annotated with information about the object on the composite image of a cropped image of an object and a background image to a V2X server (e.g., Edge, Cloud) (see 342 in FIG. 3). In step S1350, the V2X server (e.g., Edge, Cloud) can receive data annotated with information about the object on the composite image of a cropped image of an object and a background image. For example, the V2X server (e.g., Edge, Cloud) can utilize the received data as artificial intelligence training data (see 350 in FIG. 3). For example, a V2X server (e.g., Edge, Cloud) can utilize the received data for transfer learning / retraining of artificial intelligence (see 351 in Fig. 3).In step S1355, the object detector (or RSU) may receive transferred learning / retrained information (or parameters) from a V2X server (e.g., Edge, Cloud). For example, the object detector (or RSU) may perform detection of an object (e.g., V2X connected object or V2X unconnected object) based on artificial intelligence training data received from the V2X server.

[0116] FIG. 14 illustrates a method in which a first device performs wireless communication according to one embodiment of the present disclosure. The embodiment of FIG. 14 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of said embodiments may be omitted.

[0117] Referring to FIG. 14, in step S1410, the first device may acquire a first image containing a first object based on at least one sensor. In step S1420, the first device may receive a message containing information related to the first object. In step S1430, the first device may generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0118] For example, the training data may be generated based on annotating information related to the first object to a second image obtained by synthesizing the background image and the cropped image of the first object. For example, the second image may be obtained based on synthesizing the background image that does not contain the object and the cropped image of the first object. For example, the second image may be obtained based on synthesizing (i) the background image that does not contain the object and (ii) the cropped image of the first object obtained from the first image based on the fact that the first object is a V2X (vehicle-to-everything) connected object.

[0119] For example, a cropped image of the first object can be obtained by cropping the first object from the first image based on the fact that the first object is a V2X connected object.

[0120] For example, a cropped image of the first object can be obtained by excluding a non-V2X connected object among at least one object included in the first image.

[0121] For example, the training data may be generated based on the matching of the first object included in the first image and the information related to the first object included in the message. For example, whether the information related to the first object included in the first image and the information related to the first object included in the message is matched may be based on at least one of secondary inference, an N-ary tree, or a protection level related to a Global Navigation Satellite System (GNSS).

[0122] Additionally, for example, the first device may transmit the training data to the cloud or edge. For example, the training data may be used for artificial intelligence training in the cloud or edge.

[0123] For example, based on the first device detecting the first object with the at least one sensor, the training data based on the background image, the cropped image of the first object, and information related to the first object can be automatically generated.

[0124] For example, the first object included in the first image can be detected based on at least one of a bounding box or segmentation.

[0125] For example, the above message may be at least one of CPM (Collective Perception Message), CAM (Cooperative Awareness Message), or SDSM (Sensor Data Sharing Message).

[0126] The proposed method above may be applied to a device according to various embodiments of the present disclosure. First, a processor (102) of a first device (100) may acquire a first image including a first object based on at least one sensor. Then, the processor (102) of the first device (100) may control a transceiver (106) to receive a message including information related to the first object. Then, the processor (102) of the first device (100) may generate training data based on the first device, a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0127] According to one embodiment of the present disclosure, a first device configured to perform wireless communication may be provided. For example, the first device may include at least one transceiver; at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, the instructions may cause the first device, based on execution by the at least one processor: to acquire a first image including a first object based on at least one sensor; to receive a message including information related to the first object; and to generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0128] According to one embodiment of the present disclosure, a processing device configured to control a first device may be provided. For example, the processing device may include at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, the instructions may cause the first device, based on execution by the at least one processor: to acquire a first image including a first object based on at least one sensor; to receive a message including information related to the first object; and to generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0129] According to one embodiment of the present disclosure, a non-transient computer-readable storage medium recording instructions may be provided. For example, when the instructions are executed, the first device may: acquire a first image including a first object based on at least one sensor; receive a message including information related to the first object; and generate training data based on a background image, a cropped image of the first object acquired from the first image, and information related to the first object.

[0130] FIG. 15 illustrates a method in which a second device performs wireless communication according to one embodiment of the present disclosure. The embodiment of FIG. 15 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of said embodiments may be omitted.

[0131] Referring to FIG. 15, in step S1510, the second device may transmit a message containing information related to the first object to the first device. In step S1520, the second device may receive training data from the first device. In step S1530, the second device may perform artificial intelligence training based on the training data. For example, the training data may be based on a background image, a cropped image of the first object, and information related to the first object.

[0132] The proposed method above may be applied to a device according to various embodiments of the present disclosure. First, the processor (202) of the second device (200) may control a transceiver (206) to transmit a message containing information related to a first object to the first device. Then, the processor (202) of the second device (200) may control the transceiver (206) to receive learning data from the first device. Then, the processor (202) of the second device (200) may perform artificial intelligence learning based on the learning data. For example, the learning data may be based on a background image, a cropped image of the first object, and information related to the first object.

[0133] According to one embodiment of the present disclosure, a second device configured to perform wireless communication may be provided. For example, the second device may include at least one transceiver; at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, based on the instructions executed by the at least one processor, the second device may: transmit a message containing information related to a first object to a first device; receive training data from the first device; and perform artificial intelligence training based on the training data. For example, the training data may be based on a background image, a cropped image of the first object, and information related to the first object.

[0134] According to one embodiment of the present disclosure, a processing device configured to control a second device may be provided. For example, the processing device may include at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, based on the instructions executed by the at least one processor, the second device may: transmit a message containing information related to a first object to a first device; receive training data from the first device; and perform artificial intelligence training based on the training data. For example, the training data may be based on a background image, a cropped image of the first object, and information related to the first object.

[0135] According to one embodiment of the present disclosure, a non-transient computer-readable storage medium recording instructions may be provided. For example, when the instructions are executed, the second device may: transmit a message containing information related to a first object to a first device; receive training data from the first device; and perform artificial intelligence training based on the training data. For example, the training data may be based on a background image, a cropped image of the first object, and information related to the first object.

[0136] According to various embodiments of the present disclosure, by cropping only the V2X connection object from an image (or video) containing at least one object and compositing the cropped image of the V2X connection object onto a background image, the annotation of information regarding the V2X connection object included in a V2X message can be performed more precisely. Furthermore, for example, more accurate artificial intelligence training data can be generated based on the creation of a cropped image of the V2X connection object and the annotation thereof. Additionally, if the operation and / or method described above are implemented in an existing V2X infrastructure that generates a large amount of image or video data, such as an RSU, a large amount of sophisticated and accurate artificial intelligence training data can be efficiently generated and secured. In summary, object detectors utilizing artificial intelligence technologies such as machine learning or deep learning require a large amount of training data, and the data annotation process, which involves adding metadata or tagging each data point, can entail a significant amount of time and effort. Therefore, if the data collection and annotation processes are shortened or automated according to the proposal of the present disclosure, a significant amount of time and cost can be saved.

[0137] Various embodiments of the present disclosure may be combined with one another.

[0138] The following describes an apparatus to which various embodiments of the present disclosure may be applied.

[0139] Although not limited to this, the various descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document may be applied to various fields requiring wireless communication / connection (e.g., 5G) between devices.

[0140] Examples are provided in more detail below with reference to the drawings. In the following drawings and descriptions, the same reference numerals may represent the same or corresponding hardware blocks, software blocks, or function blocks unless otherwise described.

[0141] FIG. 16 illustrates a communication system (1) according to one embodiment of the present disclosure. The embodiment of FIG. 16 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods and / or operations of the embodiments may be omitted.

[0142] Referring to FIG. 16, a communication system (1) to which various embodiments of the present disclosure are applied includes a wireless device, a base station, and a network. Here, the wireless device refers to a device that performs communication using wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, the wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (eXtended Reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Thing) device (100f), and an AI device / server (400). For example, the vehicle may include a vehicle equipped with wireless communication functions, an autonomous vehicle, a vehicle capable of performing inter-vehicle communication, etc. Here, the vehicle may include an Unmanned Aerial Vehicle (UAV) (e.g., a drone) and / or an Aerial Vehicle (AV) (e.g., Advanced Air Mobility). The XR device includes an Augmented Reality (AR) / Virtual Reality (VR) / Mixed Reality (MR) device and may be implemented in the form of a Head-Mounted Device (HMD), a Head-Up Display (HUD) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a robot, etc. The portable device may include a smartphone, a smartpad, a wearable device (e.g., a smartwatch, smart glasses), a computer (e.g., a laptop, etc.). The home appliance may include a TV, a refrigerator, a washing machine, etc. The IoT device may include a sensor, a smart meter, etc. For example, a base station and a network may be implemented as a wireless device, and a specific wireless device (200a) may operate as a base station / network node to other wireless devices.

[0143] Here, the wireless communication technology implemented in the wireless devices (100a to 100f) of this specification may include LTE, NR, and 6G, as well as Narrowband Internet of Things for low-power communication. For example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology and may be implemented according to standards such as LTE Cat NB1 and / or LTE Cat NB2, but is not limited to the names mentioned above. Additionally, or generally, the wireless communication technology implemented in the wireless devices (100a to 100f) of this specification may perform communication based on LTE-M technology. For example, LTE-M technology may be an example of LPWAN technology and may be referred to by various names such as eMTC (enhanced Machine Type Communication). For example, LTE-M technology may be implemented in at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the names mentioned above. Additionally or generally, wireless communication technology implemented in the wireless devices (100a to 100f) of this specification may include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) for low-power communication, and is not limited to the names mentioned above. As an example, ZigBee technology can create personal area networks (PANs) related to small / low-power digital communication based on various standards such as IEEE 802.15.4, and may be referred to by various names.

[0144] Wireless devices (100a to 100f) can be connected to a network (300) through a base station (200). Artificial Intelligence (AI) technology may be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (400) through the network (300). The network (300) can be configured using a 3G network, a 4G (e.g., LTE) network, or a 5G (e.g., NR) network. The wireless devices (100a to 100f) may communicate with each other through the base station (200) / network (300), but they may also communicate directly (e.g., sidelink communication) without going through the base station / network. For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (Vehicle to Vehicle) / V2X (Vehicle to everything) communication). Also, IoT devices (e.g., sensors) can communicate directly with other IoT devices (e.g., sensors) or other wireless devices (100a to 100f).

[0145] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a~100f) / base station (200) and base station (200) / base station (200). Here, wireless communication / connection can be achieved through various wireless access technologies (e.g., 5G NR), such as uplink / downlink communication (150a), sidelink communication (150b) (or D2D communication), and inter-base station communication (150c) (e.g., relay, IAB (Integrated Access Backhaul)). Through wireless communication / connection (150a, 150b, 150c), wireless devices and base stations / wireless devices, and base stations and base stations can transmit / receive wireless signals to / from each other. For example, wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, based on various proposals of the present disclosure, at least some of the following may be performed: various configuration information setting processes for transmitting / receiving wireless signals, various signal processing processes (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), resource allocation processes, etc.

[0146] FIG. 17 illustrates a wireless device according to one embodiment of the present disclosure. The embodiment of FIG. 17 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of the embodiments may be omitted.

[0147] Referring to FIG. 17, the first wireless device (100) and the second wireless device (200) can transmit and receive wireless signals through various wireless access technologies (e.g., LTE, NR). Here, {the first wireless device (100), the second wireless device (200)} may correspond to {wireless device (100x), base station (200)} and / or {wireless device (100x), wireless device (100x)} of FIG. 16.

[0148] The first wireless device (100) includes one or more processors (102) and one or more memories (104), and may additionally include one or more transceivers (106) and / or one or more antennas (108). The processor (102) controls the memory (104) and / or transceivers (106) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or sequences of operation disclosed in this document. For example, the processor (102) may process information within the memory (104) to generate a first information / signal and then transmit a wireless signal containing the first information / signal through the transceiver (106). Additionally, the processor (102) may receive a wireless signal containing a second information / signal through the transceiver (106) and then store information obtained from the signal processing of the second information / signal in the memory (104). The memory (104) may be connected to the processor (102) and may store various information related to the operation of the processor (102). For example, the memory (104) may store software code containing instructions for performing some or all of the processes controlled by the processor (102) or for performing the descriptions, functions, procedures, proposals, methods, and / or operation sequence diagrams disclosed in this document. Here, the processor (102) and the memory (104) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). The transceiver (106) may be connected to the processor (102) and may transmit and / or receive wireless signals through one or more antennas (108). The transceiver (106) may include a transmitter and / or receiver. The transceiver (106) may be combined with an RF (Radio Frequency) unit. In the present disclosure, a wireless device may refer to a communication modem / circuit / chip.

[0149] The second wireless device (200) includes one or more processors (202) and one or more memories (204), and may additionally include one or more transceivers (206) and / or one or more antennas (208). The processor (202) controls the memory (204) and / or transceivers (206) and may be configured to implement the descriptions, functions, procedures, proposals, methods and / or sequences of operation disclosed in this document. For example, the processor (202) may process information within the memory (204) to generate a third information / signal and then transmit a wireless signal containing the third information / signal through the transceiver (206). Additionally, the processor (202) may receive a wireless signal containing a fourth information / signal through the transceiver (206) and then store information obtained from the signal processing of the fourth information / signal in the memory (204). Memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, memory (204) may store software code containing instructions for performing some or all of the processes controlled by the processor (202) or for performing the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document. Here, the processor (202) and memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). A transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals through one or more antennas (208). The transceiver (206) may include a transmitter and / or receiver. The transceiver (206) may be interchangeable with an RF unit. In this disclosure, a wireless device may refer to a communication modem / circuit / chip.

[0150] Hereinafter, hardware elements of the wireless device (100, 200) will be described in more detail. Although not limited thereto, one or more protocol layers may be implemented by one or more processors (102, 202). For example, one or more processors (102, 202) may implement one or more layers (e.g., functional layers such as PHY, MAC, RLC, PDCP, RRC, SDAP). One or more processors (102, 202) may generate one or more Protocol Data Units (PDUs) and / or Service Data Units (SDUs) according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document. One or more processors (102, 202) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document. One or more processors (102, 202) may generate a signal (e.g., baseband signal) containing a PDU, SDU, message, control information, data, or information according to the functions, procedures, proposals, and / or methods disclosed in this document and provide it to one or more transceivers (106, 206). One or more processors (102, 202) may receive a signal (e.g., baseband signal) from one or more transceivers (106, 206) and may obtain a PDU, SDU, message, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed in this document.

[0151] One or more processors (102, 202) may be referred to as a controller, microcontroller, microprocessor, or microcomputer. One or more processors (102, 202) may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), one or more Digital Signal Processing Devices (DSPDs), one or more Programmable Logic Devices (PLDs), or one or more Field Programmable Gate Arrays (FPGAs) may be included in one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. Firmware or software configured to perform the descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be contained in one or more processors (102, 202) or stored in one or more memories (104, 204) and driven by one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or operation sequences disclosed in this document may be implemented using firmware or software in the form of code, instructions, and / or sets of instructions.

[0152] One or more memories (104, 204) may be connected to one or more processors (102, 202) and may store various forms of data, signals, messages, information, programs, code, instructions, and / or commands. One or more memories (104, 204) may be composed of ROM, RAM, EPROM, flash memory, hard drive, registers, cache memory, computer read storage media, and / or combinations thereof. One or more memories (104, 204) may be located inside and / or outside of one or more processors (102, 202). Additionally, one or more memories (104, 204) may be connected to one or more processors (102, 202) through various technologies such as wired or wireless connections.

[0153] One or more transceivers (106, 206) may transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or operation flowcharts, etc., of this document to one or more other devices. One or more transceivers (106, 206) may receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or operation flowcharts, etc., disclosed in this document from one or more other devices. For example, one or more transceivers (106, 206) may be connected to one or more processors (102, 202) and may transmit and receive wireless signals. For example, one or more processors (102, 202) may control one or more transceivers (106, 206) to transmit user data, control information, or wireless signals to one or more other devices. Additionally, one or more processors (102, 202) may control one or more transceivers (106, 206) to receive user data, control information, or wireless signals from one or more other devices. Additionally, one or more transceivers (106, 206) may be connected to one or more antennas (108, 208), and one or more transceivers (106, 206) may be configured to transmit and receive user data, control information, wireless signals / channels, etc., as described in the descriptions, functions, procedures, proposals, methods, and / or flowcharts of operation disclosed in this document through one or more antennas (108, 208). In this document, one or more antennas may be multiple physical antennas or multiple logical antennas (e.g., antenna ports). One or more transceivers (106, 206) can convert the received wireless signal / channel, etc. from an RF band signal to a baseband signal in order to process the received user data, control information, wireless signal / channel, etc. using one or more processors (102, 202).One or more transceivers (106, 206) can convert user data, control information, wireless signals / channels, etc. processed using one or more processors (102, 202) from baseband signals to RF band signals. To this end, one or more transceivers (106, 206) may include (analog) oscillators and / or filters.

[0154] FIG. 18 illustrates a signal processing circuit for a transmission signal according to one embodiment of the present disclosure. The embodiment of FIG. 18 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, suggestions, methods, and / or operations of the embodiments may be omitted.

[0155] Referring to FIG. 18, the signal processing circuit (1000) may include a scrambler (1010), a modulator (1020), a layer mapper (1030), a precoder (1040), a resource mapper (1050), and a signal generator (1060). Although not limited thereto, the operation / function of FIG. 18 may be performed in the processor (102, 202) and / or transceiver (106, 206) of FIG. 17. The hardware elements of FIG. 18 may be implemented in the processor (102, 202) and / or transceiver (106, 206) of FIG. 17. For example, blocks 1010 through 1060 may be implemented in the processor (102, 202) of FIG. 17. Additionally, blocks 1010 to 1050 may be implemented in the processor (102, 202) of FIG. 17, and block 1060 may be implemented in the transceiver (106, 206) of FIG. 17.

[0156] The codeword can be converted into a wireless signal through the signal processing circuit (1000) of FIG. 18. Here, the codeword is an encoded bit sequence of an information block. The information block may include a transmission block (e.g., UL-SCH transmission block, DL-SCH transmission block). The wireless signal can be transmitted through various physical channels (e.g., PUSCH, PDSCH).

[0157] Specifically, a codeword can be converted into a scrambled bit sequence by a scrambler (1010). The scrambled sequence used for scrambling is generated based on an initialization value, which may include ID information of a wireless device, etc. The scrambled bit sequence can be modulated into a modulation symbol sequence by a modulator (1020). The modulation method may include pi / 2-BPSK (pi / 2-Binary Phase Shift Keying), m-PSK (m-Phase Shift Keying), m-QAM (m-Quadrature Amplitude Modulation), etc. The complex modulation symbol sequence can be mapped to one or more transmission layers by a layer mapper (1030). The modulation symbols of each transmission layer can be mapped to the corresponding antenna port(s) by a precoder (1040) (precoding). The output z of the precoder (1040) can be obtained by multiplying the output y of the layer mapper (1030) by an N*M precoding matrix W. Here, N is the number of antenna ports and M is the number of transmission layers. Here, the precoder (1040) can perform precoding after performing transform precoding (e.g., DFT transform) on the complex modulation symbols. Additionally, the precoder (1040) can perform precoding without performing transform precoding.

[0158] A resource mapper (1050) can map the modulation symbols of each antenna port to a time-frequency resource. The time-frequency resource may include multiple symbols (e.g., CP-OFDMA symbols, DFT-s-OFDMA symbols) in the time domain and multiple subcarriers in the frequency domain. A signal generator (1060) generates a radio signal from the mapped modulation symbols, and the generated radio signal can be transmitted to another device through each antenna. To this end, the signal generator (1060) may include an Inverse Fast Fourier Transform (IFFT) module, a Cyclic Prefix (CP) inserter, a Digital-to-Analog Converter (DAC), a frequency uplink converter, etc.

[0159] The signal processing process for a received signal in a wireless device can be configured as the inverse of the signal processing process (1010–1060) of FIG. 18. For example, a wireless device (e.g., 100, 200 in FIG. 17) can receive a wireless signal from the outside through an antenna port / transceiver. The received wireless signal can be converted into a baseband signal through a signal restorer. To this end, the signal restorer may include a frequency downlink converter, an analog-to-digital converter (ADC), a CP remover, and a Fast Fourier Transform (FFT) module. Subsequently, the baseband signal can be restored into a codeword through a resource de-mapper process, a postcoding process, a demodulation process, and a de-scrambling process. The codeword can be restored into the original information block through decoding. Accordingly, a signal processing circuit (not shown) for a received signal may include a signal restorer, a resource de-mapper, a postcoder, a demodulator, a de-scrambler, and a decoder.

[0160] FIG. 19 illustrates a wireless device according to one embodiment of the present disclosure. The wireless device may be implemented in various forms depending on the use-example / service (see FIG. 16). The embodiment of FIG. 19 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0161] Referring to FIG. 19, the wireless device (100, 200) corresponds to the wireless device (100, 200) of FIG. 17 and may be composed of various elements, components, units / parts, and / or modules. For example, the wireless device (100, 200) may include a communication unit (110), a control unit (120), a memory unit (130), and additional elements (140). The communication unit may include a communication circuit (112) and transceiver(s) (114). For example, the communication circuit (112) may include one or more processors (102, 202) and / or one or more memories (104, 204) of FIG. 17. For example, the transceiver(s) (114) may include one or more transceivers (106, 206) and / or one or more antennas (108, 208) of FIG. 17. The control unit (120) is electrically connected to the communication unit (110), the memory unit (130), and additional elements (140) and controls the general operation of the wireless device. For example, the control unit (120) may control the electrical / mechanical operation of the wireless device based on a program / code / command / information stored in the memory unit (130). Additionally, the control unit (120) may transmit information stored in the memory unit (130) to an external (e.g., another communication device) via a wireless / wired interface through the communication unit (110), or store information received from an external (e.g., another communication device) via a wireless / wired interface through the communication unit (110) in the memory unit (130).

[0162] The additional element (140) can be configured in various ways depending on the type of wireless device. For example, the additional element (140) may include at least one of a power unit / battery, an input / output unit (I / O unit), a driving unit, and a computing unit. Although not limited thereto, the wireless device may be implemented in the form of a robot (Fig. 16, 100a), a vehicle (Fig. 16, 100b-1, 100b-2), an XR device (Fig. 16, 100c), a portable device (Fig. 16, 100d), a home appliance (Fig. 16, 100e), an IoT device (Fig. 16, 100f), a digital broadcasting terminal, a hologram device, a public safety device, an MTC device, a medical device, a fintech device (or financial device), a security device, a climate / environment device, an AI server / device (Fig. 16, 400), a base station (Fig. 16, 200), a network node, etc. Wireless devices can be used in a movable or fixed location depending on the use—e.g., service.

[0163] In FIG. 19, various elements, components, units / parts, and / or modules within the wireless device (100, 200) may be entirely interconnected via a wired interface, or at least partially connected via a communication unit (110). For example, within the wireless device (100, 200), the control unit (120) and the communication unit (110) may be connected via a wire, and the control unit (120) and the first unit (e.g., 130, 140) may be connected wirelessly via the communication unit (110). Additionally, each element, component, unit / part, and / or module within the wireless device (100, 200) may include one or more additional elements. For example, the control unit (120) may be composed of one or more sets of processors. For example, the control unit (120) may be composed of a set of a communication control processor, an application processor, an Electronic Control Unit (ECU), a graphics processing processor, a memory control processor, etc. As another example, the memory unit (130) may be composed of RAM (Random Access Memory), DRAM (Dynamic RAM), ROM (Read Only Memory), flash memory, volatile memory, non-volatile memory and / or a combination thereof.

[0164] Hereinafter, an implementation example of FIG. 19 will be described in more detail with reference to the drawings.

[0165] FIG. 20 illustrates a portable device according to one embodiment of the present disclosure. The portable device may include a smartphone, a smartpad, a wearable device (e.g., a smartwatch, smart glasses), a portable computer (e.g., a laptop, etc.). The portable device may be referred to as a Mobile Station (MS), a User Terminal (UT), a Mobile Subscriber Station (MSS), a Subscriber Station (SS), an Advanced Mobile Station (AMS), or a Wireless Terminal (WT). The embodiment of FIG. 20 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0166] Referring to FIG. 20, the portable device (100) may include an antenna unit (108), a communication unit (110), a control unit (120), a memory unit (130), a power supply unit (140a), an interface unit (140b), and an input / output unit (140c). The antenna unit (108) may be configured as part of the communication unit (110). Blocks 110 to 130 / 140a to 140c each correspond to blocks 110 to 130 / 140 of FIG. 19.

[0167] The communication unit (110) can transmit and receive signals (e.g., data, control signals, etc.) with other wireless devices and base stations. The control unit (120) can control the components of the portable device (100) to perform various operations. The control unit (120) may include an AP (Application Processor). The memory unit (130) can store data / parameters / programs / code / commands required for the operation of the portable device (100). Additionally, the memory unit (130) can store input / output data / information, etc. The power supply unit (140a) supplies power to the portable device (100) and may include wired / wireless charging circuits, batteries, etc. The interface unit (140b) can support the connection between the portable device (100) and other external devices. The interface unit (140b) may include various ports (e.g., audio input / output ports, video input / output ports) for connection with external devices. The input / output unit (140c) can receive or output video information / signals, audio information / signals, data, and / or information input by a user. The input / output unit (140c) may include a camera, a microphone, a user input unit, a display unit (140d), a speaker and / or a haptic module, etc.

[0168] For example, in the case of data communication, the input / output unit (140c) acquires information / signals (e.g., touch, text, voice, image, video) input from the user, and the acquired information / signals can be stored in the memory unit (130). The communication unit (110) converts the information / signals stored in the memory into wireless signals and can directly transmit the converted wireless signals to another wireless device or to a base station. Additionally, the communication unit (110) can receive wireless signals from another wireless device or base station and then restore the received wireless signals to their original information / signals. The restored information / signals can be stored in the memory unit (130) and then output in various forms (e.g., text, voice, image, video, haptic) through the input / output unit (140c).

[0169] FIG. 21 illustrates a vehicle or autonomous vehicle according to one embodiment of the present disclosure. The vehicle or autonomous vehicle may be implemented as a mobile robot, vehicle, train, manned or unmanned aerial vehicle (AV), ship, etc. The embodiment of FIG. 21 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0170] Referring to FIG. 21, a vehicle or autonomous vehicle (100) may include an antenna unit (108), a communication unit (110), a control unit (120), a driving unit (140a), a power supply unit (140b), a sensor unit (140c), and an autonomous driving unit (140d). The antenna unit (108) may be configured as part of the communication unit (110). Blocks 110 / 130 / 140a to 140d each correspond to blocks 110 / 130 / 140 of FIG. 19.

[0171] The communication unit (110) can transmit and receive signals (e.g., data, control signals, etc.) with external devices such as other vehicles, base stations (e.g., base stations, roadside base stations (Roadside units), etc.), and servers. The control unit (120) can perform various operations by controlling elements of the vehicle or autonomous vehicle (100). The control unit (120) may include an Electronic Control Unit (ECU). The driving unit (140a) can drive the vehicle or autonomous vehicle (100) on the ground. The driving unit (140a) may include an engine, motor, power train, wheels, brakes, steering device, etc. The power supply unit (140b) supplies power to the vehicle or autonomous vehicle (100) and may include wired / wireless charging circuits, batteries, etc. The sensor unit (140c) can obtain vehicle status, surrounding environment information, user information, etc. The sensor unit (140c) may include an IMU (inertial measurement unit) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a position module, a vehicle forward / reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, a pedal position sensor, etc. The autonomous driving unit (140d) may implement technologies such as maintaining the driving lane, technologies for automatically adjusting speed such as adaptive cruise control, technologies for automatically driving along a predetermined path, and technologies for automatically setting a path and driving when a destination is set.

[0172] For example, the communication unit (110) can receive map data, traffic information data, etc. from an external server. The autonomous driving unit (140d) can generate an autonomous driving path and a driving plan based on the acquired data. The control unit (120) can control the drive unit (140a) so that the vehicle or the autonomous vehicle (100) moves along the autonomous driving path according to the driving plan (e.g., speed / direction control). During autonomous driving, the communication unit (110) can acquire the latest traffic information data from an external server non-periodically and can acquire surrounding traffic information data from surrounding vehicles. Additionally, during autonomous driving, the sensor unit (140c) can acquire vehicle status and surrounding environment information. The autonomous driving unit (140d) can update the autonomous driving path and the driving plan based on the newly acquired data / information. The communication unit (110) can transmit information regarding the vehicle location, autonomous driving path, driving plan, etc. to an external server. An external server can predict traffic information data in advance using AI technology, etc., based on information collected from vehicles or autonomous vehicles, and can provide the predicted traffic information data to vehicles or autonomous vehicles.

[0173] The claims described in this specification may be combined in various ways. For example, the technical features of the method claims in this specification may be combined to be implemented as a device, and the technical features of the device claims in this specification may be combined to be implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims in this specification may be combined to be implemented as a device, and the technical features of the method claims and the technical features of the device claims in this specification may be combined to be implemented as a method.

Claims

1. Regarding the method, A first device acquires a first image including a first object based on at least one sensor; The first device receives a message containing information related to the first object; and A method comprising the step of the first device generating training data based on a background image, a cropped image of the first object obtained from the first image, and information related to the first object.

2. In Paragraph 1, A method in which the above training data is generated based on annotating information related to the first object to a second image formed by synthesizing the background image and the cropped image of the first object.

3. In Paragraph 2, A method in which the second image is obtained based on compositing the background image that does not include the object and the cropped image of the first object.

4. In Paragraph 2, A method in which the second image is obtained by synthesizing (i) the background image that does not contain an object and (ii) a cropped image of the first object obtained from the first image based on the fact that the first object is a V2X (vehicle-to-everything) connected object.

5. In Paragraph 1, A method in which a cropped image of the first object is obtained by cropping the first object from the first image based on the fact that the first object is a V2X connected object.

6. In Paragraph 1, A method for obtaining a cropped image of a first object by excluding a non-V2X connected object among at least one object included in the first image.

7. In Paragraph 1, A method for generating the training data based on the matching of the first object included in the first image and the information related to the first object included in the message.

8. In Paragraph 7, A method for determining whether information related to the first object included in the first image and the first object included in the message is matched, based on at least one of secondary inference, an N-ary tree, or a protection level related to a Global Navigation Satellite System (GNSS).

9. In Paragraph 1, A method further comprising the step of the first device transmitting the training data to the cloud or edge.

10. In Paragraph 9, The above training data is used for artificial intelligence training in the cloud or the edge, a method.

11. In Paragraph 1, A method in which training data is automatically generated based on the background image, a cropped image of the first object, and information related to the first object, based on the first device detecting the first object with the at least one sensor.

12. In Paragraph 1, A method in which the first object included in the first image is detected based on at least one of a bounding box or segmentation.

13. In Paragraph 1, A method in which the above message is at least one of CPM (Collective Perception Message), CAM (Cooperative Awareness Message), or SDSM (Sensor Data Sharing Message).

14. In the first device, At least one transmitter / receiver; At least one processor; and The first device comprises at least one memory connected to the at least one processor and storing instructions, wherein the instructions are executed by the at least one processor: Based on at least one sensor, to acquire a first image including a first object; Receiving a message containing information related to the first object; and A first device that generates training data based on a background image, a cropped image of the first object obtained from the first image, and information related to the first object.

15. In a processing device configured to control a first device, At least one processor; and The first device comprises at least one memory connected to the at least one processor and storing instructions, wherein the instructions are executed by the at least one processor: Based on at least one sensor, to acquire a first image including a first object; Receiving a message containing information related to the first object; and A processing device that generates training data based on a background image, a cropped image of the first object obtained from the first image, and information related to the first object.

16. A non-transient computer-readable storage medium that records instructions, When executed, the above instructions cause the first device: Based on at least one sensor, to acquire a first image including a first object; Receiving a message containing information related to the first object; and A non-transient computer-readable storage medium that generates training data based on a background image, a cropped image of the first object obtained from the first image, and information related to the first object.

17. Regarding the method, A step in which the second device transmits to the first device a message containing information related to the first object; The second device receives training data from the first device; and The second device comprises the step of performing artificial intelligence learning based on the training data; A method based on the above training data, a background image, a cropped image of the first object, and information related to the first object.

18. In the second device, At least one transmitter / receiver; At least one processor; and The second device comprises at least one memory connected to the at least one processor and storing instructions, wherein the instructions are executed by the at least one processor: To cause the first device to transmit a message containing information related to the first object; Receiving training data from the first device; and Based on the above training data, perform artificial intelligence training, but, The above training data is based on a background image, a cropped image of the first object, and information related to the first object, a second device.

19. In a processing device configured to control a second device, At least one processor; and The second device comprises at least one memory connected to the at least one processor and storing instructions, wherein the instructions are executed by the at least one processor: To cause the first device to transmit a message containing information related to the first object; Receiving training data from the first device; and Based on the above training data, perform artificial intelligence training, but, A processing device based on the above training data, a background image, a cropped image of the first object, and information related to the first object.

20. A non-transient computer-readable storage medium that records instructions, When executed, the above commands cause the second device: To cause the first device to transmit a message containing information related to the first object; Receiving training data from the first device; and Based on the above training data, perform artificial intelligence training, but, The above training data is a non-transient computer-readable storage medium based on a background image, a cropped image of the first object, and information related to the first object.

Images