Electronic device and method for transmitting data to cluster traveling vehicles

Abstract

The invention discloses an electronic device and a method for transmitting data to a cluster driving vehicle. The electronic device comprises a camera; a communication circuit; a memory storing a neural network model and instructions; and a processor operatively connected with the camera, the communication circuitry, and the memory. When the instructions are executed by the processor, the electronic device is enabled to: acquire an image through the camera; identifying a first classification result of an object contained in the image by using the image and a neural network model; transmitting, via a communication circuit, a feature map of the neural network model based on the image to an external electronic device included in a subsequent vehicle of the vehicle on which the electronic device is mounted; acquiring a second classification result of the object calculated in the external electronic device based on the feature map from the external electronic device through the communication circuit; and deciding whether or not the vehicle travels as a preceding vehicle based on the first classification result and the second classification result.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and method for transmitting data to vehicles traveling in a group. Background Art

[0002] Generally speaking, platooning refers to the behavior of multiple vehicles driving on the road while sharing each other's driving information. Vehicles forming a platoon can travel while forming a certain formation (deformation). By reducing the distance between vehicles, platooning can reduce air resistance, thereby improving fuel efficiency and reducing the risk of accidents, while regulating traffic flow and reducing traffic congestion. The vehicles forming a platoon may include a leading vehicle and following vehicles. An electronic device arranged in the leading vehicle can control platooning. For example, the electronic device can identify an external object in front of the leading vehicle, set a driving route based on the external object, and control the speed and direction of the vehicle.

[0003] The above information may be used as background art to help understand the present disclosure. No part of the above content is claimed or identified as prior art that can be applied to the present disclosure. Summary of the Invention

[0004] According to one embodiment of the present invention, an electronic device may include: a camera; a communication circuit; a memory storing a neural network model and instructions; and a processor operatively connected to the camera, the communication circuit, and the memory. The instructions, when executed by the processor, enable the electronic device to capture an image via the camera. The instructions, when executed by the processor, enable the electronic device to use the image and the neural network model to identify a first classification result of an object contained in the image. The instructions, when executed by the processor, enable the electronic device to transmit, via the communication circuit, a feature map based on the neural network model for the image to an external electronic device contained in a subsequent vehicle in which the electronic device is mounted. The instructions, when executed by the processor, enable the electronic device to obtain, from the external electronic device, via the communication circuit, a second classification result of the object calculated in the external electronic device based on the feature map. The instructions, when executed by the processor, enable the electronic device to determine, based on the first and second classification results, whether the vehicle is traveling as a preceding vehicle.

[0005] According to an embodiment of the present invention, the electronic device may include: a camera; a communication circuit; a memory storing a neural network model and instructions; and a processor operatively connected to the camera, the communication circuit, and the memory. The instructions, when executed by the processor, enable the electronic device to acquire an image through the camera. The instructions, when executed by the processor, enable the electronic device to use the image and the neural network model to identify the area in the image that contains the vehicle ahead. The instructions, when executed by the processor, enable the electronic device to send the remaining areas in the image, except for the area, to an external electronic device contained in a subsequent vehicle on which the electronic device is mounted, through the communication circuit.

[0006] According to an embodiment of the present invention, a method performed by an electronic device including a camera and a communication circuit may include the following operations. The method may acquire an image through the camera. The method may use the image and the neural network model to identify a first classification result of an object contained in the image. The method may send, through the communication circuit, a feature map of the neural network model based on the image to an external electronic device contained in a subsequent vehicle on which the electronic device is mounted. The method may obtain, from the external electronic device, through the communication circuit, a second classification result of the object calculated in the external electronic device based on the feature map. The method may determine whether the vehicle is traveling as a leading vehicle based on the first classification result and the second classification result.

[0007] According to an embodiment of the present invention, a method performed by an electronic device including a camera and a communication circuit may include the following operations: The method may capture an image using the camera; the method may use the image and a neural network model to identify an area in the image containing a preceding vehicle; and the method may transmit, via the communication circuit, the remaining areas of the image, excluding the preceding area, to an external electronic device included in a subsequent vehicle in which the electronic device is mounted.

[0008] According to an embodiment of the present invention, a non-transitory computer readable storage medium (non-transitory computer readable storage medium) can be used to store one or more programs. When the one or more programs are executed by a processor in an electronic device having a camera and a communication circuit, the electronic device is enabled to acquire an image through the camera. When the one or more programs are executed by the processor, the electronic device is enabled to use the image and the neural network model to identify a first classification result of an object contained in the image. When the one or more programs are executed by the processor, the electronic device is enabled to send a feature map of the neural network model based on the image to an external electronic device contained in a subsequent vehicle on which the electronic device is mounted through the communication circuit. When the one or more programs are executed by the processor, the electronic device is enabled to obtain a second classification result of the object calculated in the external electronic device based on the feature map from the external electronic device through the communication circuit. When the one or more programs are executed by the processor, the electronic device is enabled to determine whether the vehicle is traveling as a leading vehicle based on the first classification result and the second classification result.

[0009] According to an embodiment of the present invention, a non-transitory computer-readable storage medium can be used to store one or more programs. When the one or more programs are executed by a processor in an electronic device having a camera and a communication circuit, the electronic device can acquire an image through the camera. When the one or more programs are executed by the processor, the electronic device can use the image and a neural network model to identify the area in the image that contains the vehicle ahead. When the one or more programs are executed by the processor, the electronic device can send the remaining areas in the image except the area to an external electronic device contained in a subsequent vehicle on which the electronic device is mounted through the communication circuit.

[0010] The electronic device or method according to the embodiment of the present invention can reduce the resources required for data exchange between clustered vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 FIG. 1 is a block diagram of an electronic device for cluster driving of vehicles according to an embodiment.

[0012] Figure 2 A schematic diagram shows platooning vehicles equipped with electronic devices according to an embodiment.

[0013] Figure 3A front image according to an embodiment is shown.

[0014] Figure 4 A neural network for generating feature maps according to one embodiment is shown.

[0015] Figure 5 A schematic diagram shows a group of vehicles traveling in which a leading vehicle is replaced according to an embodiment.

[0016] Figure 6a The diagram schematically illustrates how a rear vehicle identifies a dangerous object during cluster driving according to an embodiment.

[0017] Figure 6b A front image according to an embodiment is shown.

[0018] Figure 7 An example of an area in a front image that needs to be transmitted to a rear vehicle according to an embodiment is shown.

[0019] Figure 8 An example of a method for calculating the width of an object in a front image according to an embodiment is shown.

[0020] Figure 9 An example of a method for calculating the size of an object according to an embodiment is shown.

[0021] Figure 10 is an exemplary flow chart of an operating method of an electronic device according to an embodiment.

[0022] Figure 11 is an exemplary flow chart of an operating method of an electronic device according to an embodiment.

[0023] Figure 12 An example block diagram of an autonomous driving system for a vehicle according to an embodiment is shown.

[0024] Figure 13 and Figure 14 An example block diagram of an autonomous driving mobile body according to an embodiment is shown.

[0025] Figure 15 An example of a gateway is shown in relation to a user device of various embodiments.

[0026] Figure 16 is a schematic diagram for illustrating the operation of an electronic device for training a neural network based on a training data set according to an embodiment.

[0027] Figure 17 is a block diagram of an electronic device according to an embodiment.

[0028] Figure 18 This is a schematic diagram illustrating a state where a tractor and trailer are not connected.

[0029] Figure 19 This is a schematic diagram illustrating the connection state between a tractor and a trailer. DETAILED DESCRIPTION

[0030] The embodiments of the present specification will be described below with reference to the accompanying drawings. In the description of the drawings, similar or related components may be represented by similar reference symbols.

[0031] Over the years, the trucking industry has experienced sustained growth and expanded its service offerings to address more complex supply chains. These services include last-mile deliveries, drop-trailer programs, and intermodal transportation through ports (a form of transportation in which freight is delivered to its destination using two or more different modes of transportation, such as ship and rail, or ship and aircraft).

[0032] As a result, due to the extremely diverse modes of cargo transportation, manufacturers of freight equipment have designed different forms of equipment to transport cargo according to various transportation needs.

[0033] In this specification, a truck that tows a trailer whose main purpose is to transport (carry or cater) freight will be generally referred to as a tractor.

[0034] The tractors described in this specification can be divided into conventional trucks (or bonneted trucks), cab-over trucks (or cab-over engine trucks), and semi-conventional trucks, which are between conventional trucks and cab-over trucks, based on the position and shape of their cabs.

[0035] In a conventional truck, the engine and hood are located above the front axle in front of the tractor cab, with the driver sitting behind the front axle. This type of tractor, with the engine located in front of the driver, is primarily used in North America.

[0036] In contrast, a cab-over truck has the cab at the front of the tractor, with the driver seated in front of the front axle. The front of the tractor is flat, often called a "flat face" or "flat nose," with the engine positioned below the driver. This type of tractor is primarily used in most countries in Europe and Asia.

[0037] Just as tractors come in many forms depending on their purpose and needs, the trailers towed by tractors also come in a variety of styles. The most representative trailer types include full-trailers and semi-trailers. The difference between full-trailers and semi-trailers lies in whether the trailer has both a front axle and a rear axle. These trailers can be connected to a box truck or tractor using a coupling device.

[0038] Specifically, a full-trailer is a commercial freight trailer equipped with a front axle and a rear axle. Designed to carry its total weight independently of a towing vehicle, a full-trailer is equipped with a drawbar for connecting to a hauling unit or towing unit, such as a tractor. This type of trailer is widely used in the United States, Canada, and other regions.

[0039] In contrast, a semi-trailer is a cargo trailer equipped with only a rear axle and no front axle. A large portion of its weight is supported by a tractor vehicle connected to it by a hitch called a fifth wheel. When the semi-trailer is detached from the tractor vehicle and stationary, the weight of the trailer can be supported by the landing gear mounted on the bottom of the semi-trailer, which is vertically extended to the ground. The combination of a semi-trailer and a tractor vehicle is called a semi-trailer truck, and in the United States it is often referred to as a "semi-trailer", "tractor-trailer", "semi-truck", "big rig" or "semi". The "fifth wheel" mentioned above refers to a horizontal wheel mounted on the axle of a tractor truck to facilitate steering of the trailer. It is also called the fifth wheel. A "fifth wheel" is a device used to achieve a movable connection (movable connection) between the tractor and the semi-trailer. It typically consists of a trunnion plate and a locking device that securely fastens the kingpin mounted on the semi-trailer to the trunnion plate on the tractor.

[0040] In this specification, the following terms will be used based on the above-mentioned tractor / trailer. For convenience of explanation, a "trailer" refers to a cargo transport vehicle connected to a tractor for a trailer, and a "tractor" refers to the towing vehicle used to move the trailer. Furthermore, to minimize limitations on the scope of the present invention due to the embodiments described in the detailed description, a tractor hauling / towing a "trailer" may be described as a "towing vehicle," and a trailer towed by the tractor may be described as a "towed vehicle." These terms may be used interchangeably in the description.

[0041] In addition, for the convenience of explanation, it is preferred that the “trailer” mentioned in this specification be understood to refer to a “semi-trailer”, but not limited thereto.

[0042] Figure 1 FIG. 1 is a block diagram of an electronic device for cluster driving of vehicles according to an embodiment. Figure 2 A schematic diagram shows platooning vehicles equipped with electronic devices according to an embodiment.

[0043] Reference Figure 1 According to an embodiment, the electronic device 101 may include a communication circuit 211, a processor 221, a memory 231 and / or a camera 241. According to an embodiment, the electronic device 101 may include a GPS (Global Positioning System) sensor.

[0044] In one embodiment, the electronic device 101 may correspond to an ECU (electronic control unit: electronic control unit) in a vehicle or be included in an ECU. The ECU may be referred to as an ECM (electronic control module: electronic control module). The electronic device 101 may be configured as independent hardware (Hardware) for providing the functions according to an embodiment of the present invention in a vehicle. However, the embodiment is not limited thereto, and the electronic device 101 may correspond to a device attached to a vehicle (e.g., a driving recorder) or be included in the device. In one embodiment, the electronic device 101 may be referred to as being arranged in a leading vehicle (e.g., Figure 2 An electronic device within a vehicle 210).

[0045] For example, the processor 221, the memory 231, the communication circuit 211, and / or the camera 241 may be electrically and / or operatively coupled to each other via electronic components such as a communication bus 251. Hereinafter, the operative coupling of hardware may refer to establishing a direct or indirect connection between hardware via a wired or wireless manner, such that a first hardware component can control a second hardware component. Although illustrated as different modules, the embodiments are not limited thereto. Figure 1 Some of the hardware in the electronic device 101 may be included in a single integrated circuit such as a SoC (system on a chip). The type and / or quantity of hardware included in the electronic device 101 is not limited to Figure 1 For example, the electronic device 101 may only include Figure 1 Some of the hardware shown.

[0046] According to one embodiment, the communication circuit 211 of the electronic device 101 may include hardware components for supporting the transmission and / or reception of electrical signals between the electronic device 101 and the external electronic devices 103 and 105. The communication circuit 211 may include, for example, at least one of a modem, an antenna, and an optical / electronic converter (O / E). The communication circuit 211 may support the transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, local area network (LAN), wide area network (WAN), WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), ZigBee, LTE (long term evolution), 5G NR (new radio), non-terrestrial networks, and / or 6G.

[0047] According to one embodiment, the communication circuit 211 can be used to wirelessly communicate with the external electronic devices 103 and 105. For example, the electronic device 101 can wirelessly communicate with the external electronic devices 103 and 105 and / or other external electronic devices (e.g., base stations and / or satellites) through the communication circuit 211. The communication circuit 211 can be connected to an antenna (e.g., a base station) for sending and / or receiving signals. Figure 14 The communication circuit 211 can convert the analog signal provided by the processor 221 into a digital signal and upconvert the baseband signal into a radio frequency (RF) signal. The electronic device 101 can obtain information related to the real-time position of the cluster through the GPS sensor 150 and send the data containing the information to the external electronic devices 103 and 105 through the communication circuit 211. The electronic device 101 can send a signal for controlling subsequent vehicles (for example: Figure 2 The external electronic devices 103 and 105 can receive the signal through the communication circuits 213 and 215.

[0048] According to one embodiment, the electronic device 101 may include hardware for processing data based on one or more instructions. The hardware for processing data may include a processor 120. For example, the hardware for processing data may include an ALU (arithmetic and logic unit), an FPU (floating point unit), an FPGA (field programmable gate array), a CPU (central processing unit), and / or an AP (application processor). The processor 120 may have a single-core processor architecture, or a multi-core processor architecture, such as a dual-core, quad-core, hexa-core, or octa-core processor.

[0049] According to one embodiment, the memory 231 may include a hardware component for storing data and / or instructions that can be executed by the processor 221. For example, the memory 231 may include a volatile memory such as RAM (random-access memory) and / or a non-volatile memory such as ROM (read-only memory). For example, the volatile memory may include at least one of DRAM (dynamic random access memory), SRAM (static random access memory), cache RAM, and PSRAM (pseudo-static random access memory). For example, the non-volatile memory may include at least one of PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), flash memory, a hard disk, an optical disk, an SSD (solid state drive), and an eMMC (embedded multi-media card).

[0050] According to one embodiment, the memory 231 of the electronic device 101 may include an image processing model (eg, a neural network model) (eg, Figure 4The electronic device 101 may recognize the external object 270 based on the image processing model (e.g., neural network model) stored in the memory 231. Hereinafter, the image processing model may be referred to as the neural network model 400. According to one embodiment, the neural network model 400 may include a mathematical model of biological neural activities related to reasoning and / or recognition and / or hardware (e.g., CPU, GPU (graphic processing unit) and / or NPU (neural processing unit)), software, or any combination thereof for driving the mathematical model. The neural network model 400 may be based on CNN (convolutional neural network) and / or LSTM (long-short term memory), but is not limited thereto. The neural network model 400 may include at least one of a GAN (generative adversarial networks) model, a RESnet (residual network) model, a NAFnet (nonlinear-activation-free network) model, or a Text-Prior-based Super-resolution model.

[0051] According to one embodiment, the camera 241 may include a lens assembly or an image sensor. The lens assembly may collect light emitted from a subject serving as an image capture object. The lens assembly may include one or more lenses. For example, the camera 241 may include multiple lens assemblies. For example, among the multiple lens assemblies of the camera 241, some lens assemblies may have the same lens properties (e.g., viewing angle, focal length, autofocus, aperture value (fnumber), or optical zoom), or at least one lens assembly may have one or more lens properties that are different from those of the other lens assemblies. The lens assembly may include a wide-angle lens or a telephoto lens. For example, the electronic device 101 may include a flash for the camera 241. The flash may include one or more light-emitting diodes (e.g., RGB (red-green-blue) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs) or a xenon lamp. For example, the image sensor may obtain an image corresponding to the subject by converting light emitted from or reflected from the subject and transmitted through the lens assembly into an electrical signal. According to one embodiment, the image sensor may include one image sensor selected from sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an infrared (IR) sensor, or an ultraviolet (UV) sensor; multiple image sensors having the same properties; or multiple image sensors having different properties. Each image sensor included in the image sensor may be implemented, for example, using a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0052] According to one embodiment, the camera 241 may be positioned (or arranged) facing a certain direction of the vehicle 210 . Figure 2 The electronic device 101 (or the camera 241 included in the electronic device 101) is shown as being arranged facing the front (front direction and / or driving direction) of the vehicle 210, but the present invention is not limited thereto. For example, the camera 241 of the electronic device 101 may be arranged facing at least one of the rear (rear direction) or side (side direction) of the vehicle 210.

[0053] According to one embodiment, the electronic device 101 may identify the surrounding environment of the leading vehicle (e.g., vehicle 210) through the camera 241. For example, the electronic device 101 may identify an external object 270 based on an image acquired by the camera 241. For example, the electronic device 101 may utilize the neural network model 400 to identify an external object 270 corresponding to the image acquired by the camera 241.

[0054] Each of the electronic devices 103 and 105 may include substantially the same components as the electronic device 101. For example, each of the electronic devices 103 and 105 may include a processor 223 and 225, a memory 233 and 235, a communication circuit 213 and 215, a camera 243 and 245, and / or a communication bus 253 and 255. The above description of the components of the electronic device 101 may also be substantially applicable to the components of the electronic devices 103 and 105. For example, the memory 223 and 225 may include an image model.

[0055] According to one embodiment, each of the electronic devices 103 and 105 may correspond to an ECU (Electronic Control Unit) in a vehicle or be included in an ECU. According to one embodiment, each of the electronic devices 103 and 105 may correspond to a device attached to a vehicle (e.g., a driving recorder) or be included in the device. According to one embodiment, each of the electronic devices 103 and 105 may be referred to as a device disposed on a subsequent vehicle (e.g., Figure 2 Electronic devices within vehicles 230, 250).

[0056] Reference Figure 2 , two or more vehicles 210, 230, 250 forming a platoon can travel while maintaining a specified formation. Each of the vehicles 210, 230, 250 may include an electronic device for platoon travel (e.g., Figure 1The electronic devices 101, 103, 105 in the vehicle 210, 230, 250 can use wireless communication technology to share control information of the vehicle 210, 230, 250 and information collected by the electronic devices 101, 103, 105 arranged on the vehicle 210, 230, 250 in real time. The wireless communication technology used to exchange information between the electronic devices 101, 103, 105 may include V2X (Vehicle to Everything), such as V2I (Vehicle to Infrastructure), V2D (Vehicle to Device), V2V (Vehicle to Vehicle), V2P (Vehicle to Pedestrian), and various wireless access technologies (Wireless Access Technologies) such as cellular 5G NR (New Radio) Sidelink and 802.11-based short-range dedicated communication (Dedicated Short Range Communication: DSRC).

[0057] Vehicles 210, 230, 250 can be divided into leading vehicles and following vehicles. The leading vehicle can refer to the vehicle located at the front of the cluster of vehicles 210, 230, 250, and the following vehicles can refer to the remaining vehicles except the leading vehicle. The electronic device 101 arranged in the leading vehicle 210 can be used to control the overall operation of the cluster driving. For example, because the leading vehicle is located at the front of the cluster, it can include more electronic components (such as radar (radio detection and ranging), laser radar (light detection and ranging), camera) than other following vehicles. However, it is not limited to this.

[0058] The electronic device 101 can transmit and / or receive data with other electronic devices (e.g., base stations and / or satellites). For example, in order to determine a driving route, the electronic device 101 can receive data containing information related to a driving route from an external electronic device, and send data containing information related to the real-time location of the cluster to the external electronic device.

[0059] The electronic device 101 can control the driving of the vehicles 210, 230, 250 based on information related to the vehicles 210, 230, 250 traveling in the cluster (e.g., driving routes, driving speeds, spacing between vehicles 210, 230, 250, and / or the formation of the cluster) and / or information related to the surrounding environment. For example, the electronic device 101 can transmit signals for controlling the driving of the cluster to the electronic devices 103, 105 respectively arranged in the following vehicles 230, 250. The electronic devices 103, 105 can control the driving of the following vehicles (e.g., vehicles 230, 250) based on the signals received from the electronic device 101.

[0060] Information related to the surrounding environment may include information related to external objects 270. For example, external objects 270 are objects located around the driving routes of vehicles 210, 230, and 250, and may be objects that need to be considered for safe driving. For example, external objects 270 may include other vehicles, lane lines, lanes, traffic lights, crosswalks, pedestrians, animals, and / or speed bumps, but are not limited thereto.

[0061] According to one embodiment, since the electronic device 101 disposed in the lead vehicle 210 at the front of the queue is located at the front end of the cluster, it can be configured to provide multiple functions. In addition, since the electronic device 101 provides multiple functions, the amount of information processed by the electronic device 101 and the operations performed by the electronic device 101 may be large. For example, when the electronic device 101 processes various information, if the recognition of external objects obtained from the front image via the camera 241 is incorrect, it may cause an accident to occur with the vehicles 210, 230, and 250. Therefore, a method may be needed to replace the vehicle 210, 230, and 250 with a vehicle that has a higher accuracy in external object recognition with the lead vehicle.

[0062] According to one embodiment, the vehicle 210 as the leading vehicle may send data containing information related to the surrounding environment to the vehicles 230 and 250. Therefore, when the electronic device 101 of the leading vehicle only sends the classification result (or recognition result) of its neural network model 400 and / or the instructions based on the classification result (or recognition result) (e.g., evasive maneuvers, etc.) to the electronic devices 103 and 105 of the following vehicles, it may be difficult to correct the errors of the electronic device 101 of the leading vehicle. Therefore, a method may be needed to efficiently transmit the information required for the following vehicles to independently identify the objects in the field of view of the leading vehicle. The following will refer to Figures 3 to 5 , describing an operation of replacing a vehicle having a relatively high external object recognition accuracy among the vehicles 210 , 230 , and 250 with a leading vehicle.

[0063] According to one embodiment, the electronic devices 101, 103, and 105 can send and receive data containing information related to the surrounding environment to each other. When the electronic devices 101, 103, and 105 sequentially transmit the data containing information related to the surrounding environment to the adjacent electronic devices, there may be a time difference in the transmission of the information. Therefore, in order to perform operations such as avoiding external objects, a method may be required to preferentially share the information related to the surrounding environment obtained by the cameras 241, 243, and 245 included in the electronic devices 101, 103, and 105 with the electronic devices in need. Figure 6a and Figure 6b , describes an operation for prioritizing the delivery of data containing information about the surrounding environment.

[0064] According to one embodiment, when the data size of the surrounding environment related information between the electronic devices 101, 103, and 105 is large, the consumption of communication resources may increase. Therefore, a method may be needed to efficiently reduce the size of the data including the surrounding environment related information. Figure 7 , describes the operation of reducing the data containing information about the surrounding environment.

[0065] Figure 3 A front image according to an embodiment is shown. Figure 4 A neural network for generating feature maps according to one embodiment is shown. Figure 5 A schematic diagram shows a group of vehicles traveling in which a leading vehicle is replaced according to an embodiment.

[0066] Figures 3 to 5 You can refer to Figure 1 and Figure 2 Provide explanation.

[0067] In one embodiment, referring to Figure 3 , the processor 221 may use the camera 241 to obtain an image 300. According to one embodiment, the processor 221 of the electronic device 101 may use the camera 241 to obtain an image 300 outside the vehicle 210. The electronic device 101 may use the image 300 obtained by the camera to obtain an image 300 including external objects 270 within a field of view (FoV) of the camera 241.

[0068] According to one embodiment, referring to Figure 4, the processor 221 can use the specified neural network model 400 to identify the object 270 contained in the image 300 and the type (or category) of the object 270. For example, the specified neural network model 400 can be composed of a combination of a convolution layer 410 and a fully connected layer 420, but is not limited thereto. In one embodiment, the processor 221 can identify a specified type of object (e.g., a vehicle, a traffic sign, a road marking) from the image 300 through an image segmentation model and / or an object detection (OD) model.

[0069] In one embodiment, convolution layer 410 can be used to preserve spatial information of image 300 and extract features. In one embodiment, fully connected layer (FC layer) 420 can be used to output data within a specified range. In one embodiment, convolution layer 410 can have, for example, 13 layers. In one embodiment, fully connected layer 420 can have fewer layers than convolution layer 410, for example, having only 3 layers.

[0070] In one embodiment, the image 300 can be set as the input data of the convolution layer 410. For example, the image 300 can be a frame of the image information obtained by the camera 241. The image 300 can be composed of a three-dimensional vector of 224×224×3. Based on the output data of the convolution layer 410, a three-dimensional vector of 7×7×512 can be identified. The three-dimensional vector of 7×7×512 can be set as the input data of the fully connected layer (FC layer) 420. Through the fully connected layer 420, an object value 452 representing the type (or category) of the object 270 can be output. But it is not limited to this. The object value 452 can represent the position of the object 270 in the image 300.

[0071] In one embodiment, the processor 221 may classify the type of the object 270 based on the object value 452. In one embodiment, the processor 221 may classify the type corresponding to the maximum probability value of the object value 452 from the classifiable types using the neural network model 400 as the type of the object 270.

[0072] According to one embodiment, the processor 221 may set only a specific portion (or region of interest (ROI)) of the image 300 as input data for the neural network model 400. In one embodiment, the processor 221 may obtain the region (or bounding box) where the object 270 is located using an image segmentation model. In one embodiment, the processor 221 may crop the region (or bounding box) where the object 270 is located from the image 300. In one embodiment, the processor 221 may set the cropped region (or bounding box) (or ROI) where the object 270 is located as input data for the convolution layer 410.

[0073] In one embodiment, the processor 221 may send data to the electronic devices 103, 105 via the communication circuit 211. In one embodiment, the data sent to the electronic devices 103, 105 may include information related to the surrounding environment. The information related to the surrounding environment may include information related to the external object 270. For example, the external object 270 may be an object located around the driving path of the vehicle 210, 230, 250, and may be an object that needs to be considered for safe driving. For example, the external object 270 may include other vehicles, lane lines, lanes, traffic lights, crosswalks, pedestrians, animals and / or speed bumps, but is not limited thereto. In one embodiment, the information related to the external object 270 may represent the type and / or location of the external object 270 identified by the neural network model 400. In one embodiment, the information related to the external object 270 may include information about the width and / or size of the external object 270. In one embodiment, the operation for obtaining the width and / or size information of the external object 270 may refer to Figure 8 and Figure 9 Provide explanation.

[0074] In one embodiment, the processor 221 may send the intermediate calculation result (or feature map) of the convolution layer 410 as data to the electronic devices 103 and 105 through the communication circuit 211. For example, the intermediate calculation result (or feature map) may be the calculation result of a certain layer of the convolution layer 410. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 128×128×128. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 56×56×256. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 28×28×512. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 14×14×512. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 7×7×512. But the present invention is not limited thereto.

[0075] In one embodiment, electronic devices 103 and 105 can obtain an object value representing the type (or category) of object 270 by inputting the intermediate calculation results into the neural network model 400 stored in their respective memories 233 and 235. In one embodiment, the object value may be the same as or different from the object value 452. In one embodiment, electronic devices 103 and 105 can classify the type of object 270 based on the object value. In one embodiment, electronic devices 103 and 105 can classify the type of object 270 as the type represented by the maximum probability value among the probabilities corresponding to each object value from the types classifiable by neural network model 400. In one embodiment, the classification results of object 270 by each electronic device 103 and 105 can be the same as or different from the classification result of object 270 by electronic device 101.

[0076] In one embodiment, the electronic devices 103 and 105 may transmit information about the object value 452 and / or the type (or category) (or classification result) of the object 270 identified based on the object value 452 to the electronic device 101 through the communication circuits 213 and 215 .

[0077] In one embodiment, the processor 221 of the electronic device 101 may decide whether to change the classification result of the object 270 based on the object value 452 based on the object value 452 obtained from the electronic devices 103 and 105 and / or information related to the type (or category) of the object 270 identified based on the object value 452.

[0078] In one embodiment, processor 221 may determine whether to change the classification result of object 270 based on the type of object 270 classified by the majority of electronic devices among the classification results of electronic devices 101, 103, and 105. Hereinafter, the type of object 270 classified by the majority of electronic devices may be referred to as the majority classification result (or correct answer). Hereinafter, classification results other than the majority classification result (or correct answer) may be referred to as the minority classification result (or incorrect answer).

[0079] In one embodiment, the processor 221 may not change the classification result of the object 270 based on the object value 452 being the majority classification result (or the correct answer). In one embodiment, the processor 221 may change the classification result of the object 270 based on the object value 452 being the minority classification result (or the incorrect answer). In one embodiment, the processor 221 maintaining the type (or classification result) of the object 270 unchanged may indicate that the classification result of the processor 221 is accurate. In one embodiment, the processor 221 changing the type (or classification result) of the object 270 may indicate that the classification result of the processor 221 is inaccurate.

[0080] In one embodiment, the processor 221 may identify the accuracy of the classification results of the object 270. In one embodiment, the accuracy may be calculated based on the correct answers and / or incorrect answers in the classification results of the object 270 by the processor 221 while the vehicle 210 is traveling. For example, the accuracy may represent the proportion of correct answers in the classification results of the object classified by the processor 221.

[0081] In one embodiment, processor 221 may determine whether to replace the lead vehicle based on the accuracy of electronic device 101. In one embodiment, processor 221 may determine to replace the lead vehicle when the accuracy of electronic device 101 is lower than a baseline accuracy (e.g., 95%). In another embodiment, processor 221 may determine to replace the lead vehicle when the accuracy of electronic device 101 is lower than the accuracy of electronic devices 103 and 105 included in the following vehicles.

[0082] In one embodiment, the processor 221 may select a new leading vehicle based on the correctness (or accuracy) of the electronic devices 103, 105 included in the following vehicles. For example, the processor 221 may select the vehicle equipped with the electronic device 103, 105 with the highest correctness (or accuracy) as the new leading vehicle.

[0083] In one embodiment, the processor 221 may notify (or send) information of the lead vehicle change to the electronic devices 103 and 105 through the communication circuit 211 based on the new lead vehicle being selected (or determined).

[0084] Reference Figure 5 Based on the change in the lead vehicle from vehicle 210 to vehicle 230, vehicle 210 may temporarily leave the cluster formation. In one embodiment, vehicle 210 may move to a position behind the lead vehicle. In another embodiment, vehicle 210 may move to the last position in the cluster formation, but is not limited thereto.

[0085] As described above, instead of sharing the entire image 300 with the following vehicle, the electronic device 101 can share smaller data (e.g., the intermediate calculation results of the convolution layer 410) than the entire image 300. This can reduce the resources (e.g., communication resources) required for data exchange between the electronic devices 101, 103, and 105.

[0086] Furthermore, the electronic device 101 can obtain classification results of data (e.g., intermediate calculation results of the convolutional layer 410) from the electronic devices 103 and 105, enabling the cluster to collectively calculate external objects. Thus, the electronic device 101 can achieve more accurate recognition of external objects.

[0087] Furthermore, by replacing the vehicle equipped with the electronic device with the highest accuracy rate among the electronic devices 101, 103, and 105 as the lead vehicle, the recognition reliability of the lead vehicle can be maintained at a high level. Furthermore, when the accuracy rate of the lead vehicle decreases, the recognition reliability of the lead vehicle can be maintained at a high level by replacing the lead vehicle.

[0088] Figure 6a The diagram schematically illustrates how a rear vehicle identifies a dangerous object during cluster driving according to an embodiment. Figure 6b A front image according to an embodiment is shown.

[0089] Reference Figure 6a , two or more vehicles 210, 230, 250 forming a group can travel while maintaining a specified formation. Each of the vehicles 210, 230, 250 may include an electronic device for group travel (e.g.: Figure 1 The electronic devices 101, 103, 105 in the vehicle 210, 230, 250 can use V2X communication technology to share control information of the vehicle 210, 230, 250 and information collected by the electronic devices 101, 103, 105 arranged on the vehicle 210, 230, 250 in real time.

[0090] In one embodiment, the electronic devices (e.g., electronic devices 103 and 105) included in the subsequent vehicles can send the surrounding environment related information obtained by the cameras 243 and 245 to the other electronic devices 101, 103, and 105. For example, the electronic device 103 can send the surrounding environment related information to the electronic devices 101 and 105 arranged in the other vehicles 210 and 250, respectively. The electronic devices 101 and 105 can control the driving of the vehicles 210 and 250 based on the surrounding environment related information received from the electronic device 103. For example, the electronic device 105 can send the surrounding environment related information to the electronic devices 101 and 103 arranged in the other vehicles 210 and 230, respectively. The electronic devices 101 and 103 can control the driving of the vehicles 210 and 230 based on the surrounding environment related information received from the electronic device 105.

[0091] In one embodiment, the surrounding environment-related information may include information related to external objects 270. For example, external objects 270 may be objects located around the driving path of vehicles 210, 230, and 250, and may be objects that need to be considered for safe driving. For example, external objects 270 may include other vehicles, lane lines, lanes, traffic lights, crosswalks, pedestrians, animals, and / or speed bumps, but are not limited thereto. In one embodiment, external objects 270 may be objects that are not recognized by the leading vehicle. However, this is not limited to the above. External objects 270 may also be objects that have been recognized by the leading vehicle. For example, external objects 270 may be objects whose risk level increases after being recognized by the leading vehicle.

[0092] In one embodiment, referring to Figure 6b , the processor 225 may use the camera 245 to obtain an image 600. According to one embodiment, the processor 225 of the electronic device 105 may use the camera 245 to obtain an image 600 outside the vehicle 250. The electronic device 105 may use the image 600 obtained by the camera to obtain an image 600 including external objects 690 within the field of view (FoV) of the camera 245.

[0093] According to one embodiment, the processor 225 may utilize a specified neural network model (e.g., Figure 4 The neural network model 400 (e.g., the neural network model 400) identifies the object 690 contained in the image 600 and the type (or category) of the object 690. In one embodiment, the processor 225 can identify a specified type of object (e.g., a vehicle, a traffic sign, a road marking) from the image 600 using an image segmentation model and / or an object detection model.

[0094] In one embodiment, processor 225 may classify the type of object 690 based on the object value in image 600 identified by neural network model 400. In one embodiment, processor 225 may classify the type corresponding to the maximum probability value among the types that neural network model 400 can classify as the type of object 690. However, the present invention is not limited thereto. In one embodiment, processor 225 may set only a specific portion (or region of interest (ROI)) of image 600 as input data for neural network model 400.

[0095] In one embodiment, the processor 225 may send data to the electronic devices 101 and 103 via the communication circuit 215. In one embodiment, the data sent to the electronic devices 101 and 103 may include information related to the surrounding environment. The information related to the surrounding environment may include information related to the external object 690 (e.g., type, location, and / or size).

[0096] In one embodiment, the processor 225 may send data containing information related to the surrounding environment to the electronic devices 101 and 103 based on the priority of the electronic devices 101 and 103. In one embodiment, the priority between the electronic devices 101 and 103 may be determined based on the risk level brought by the external object 690. For example, among the electronic devices 101 and 103, the electronic device with the highest risk level caused by the external object 690 may have the highest priority. For example, among the electronic devices 101 and 103, the electronic device with the lowest risk level caused by the external object 690 may have the lowest priority. For example, the risk level of the external object 690 may be the possibility of collision with the electronic device due to the position, moving direction and / or moving speed of the external object 690. For example, the higher the possibility of collision, the higher the risk level may be assessed to be.

[0097] In one embodiment, the processor 225 may sequentially send data containing information related to the surrounding environment to the electronic devices 101 and 103 based on priority. For example, the processor 225 may prioritize sending data containing information related to the surrounding environment to the electronic device 103 in the vehicle 230 that has a high probability of colliding with the object 690. Subsequently, the processor 225 may send data containing information related to the surrounding environment to the electronic device 101 in the vehicle 210 that has a relatively low probability of colliding with the object 690.

[0098] As described above, electronic device 105 can preferentially share the acquired surrounding environment related information with electronic devices that need the information (e.g., electronic device 103). Thus, by preferentially sending necessary information to electronic devices that require an immediate response, the possibility of collision can be reduced through evasive maneuvers by the electronic devices.

[0099] Figure 7 An example of an area in a front image that needs to be transmitted to a rear vehicle according to an embodiment is shown.

[0100] In one embodiment, the electronic devices 101 , 103 , and 105 may acquire images through the cameras 241 , 243 , and 245 .

[0101] In one embodiment, the image captured by the electronic device 101 mounted on the leading vehicle can be shared (or transferred) (or sent) to the electronic devices 103 and 105 of the following vehicles. In one embodiment, the feature map of the image captured by the electronic device 101 mounted on the leading vehicle (or the output result of the neural network model 400) can be shared (or transferred) (or sent) to the electronic devices 103 and 105 of the following vehicles.

[0102] In one embodiment, the image captured by the electronic device 103, 105 mounted on the subsequent vehicle can be shared (or transferred) (or sent) to the electronic device of the subsequent vehicle. In one embodiment, the feature map of the image captured by the electronic device 103, 105 mounted on the subsequent vehicle (or the intermediate calculation result of the neural network model 400) (or the output result of the neural network model 400) can be shared (or transferred) (or sent) to the electronic device of the subsequent vehicle. For example, the image captured by the electronic device 103 can be shared with the electronic device 105 of the subsequent vehicle. For example, the feature map of the image captured by the electronic device 103 (or the output result of the neural network model 400) can be shared (or transferred) (or sent) to the electronic device 105 of the subsequent vehicle.

[0103] In one embodiment, the electronic device 103 or 105 mounted on the following vehicle may share (or transmit) a portion of the captured image to the electronic device of the following vehicle. In one embodiment, the portion may be an area of ​​the image captured by the electronic device 103 or 105 that does not include the preceding vehicle.

[0104] For example, refer to Figure 7 , the electronic device 103 can recognize the image 700 through the camera 243. The electronic device 103 can identify an external object corresponding to the front vehicle 210 in the image 700. The electronic device 103 can identify the category (or classification) of the external object corresponding to the front vehicle 210. Based on the recognition of the external object corresponding to the front vehicle 210, the electronic device 103 can determine the area 720 corresponding to the external object. For example, the area 720 can include first vertices X1, X2, X3, and X4. For example, the electronic device 103 can identify the respective coordinates of the first vertices X1, X2, X3, and X4 of the area 720. For example, the coordinates of the first point X1 can be (x11, y11). For example, the coordinates of the second point X2 can be (x21, y21). For example, the coordinates of the third point X3 can be (x31, y31). For example, the coordinates of the fourth point X4 can be (x41, y41).

[0105] In one embodiment, the electronic device 103 may identify areas 730 and 735 in the image 700, excluding the area 720 corresponding to the external object. In one embodiment, the areas 730 and 735 may include: the area 730 to the left of the left vertical line connecting the first point X1 and the third point X3 of the area 720, and the area 735 to the right of the right vertical line connecting the second point X2 and the fourth point X4 of the area 720. However, this is not limiting. In one embodiment, the areas may include: the area above the upper horizontal line connecting the first point X1 and the second point X2 of the area 720, and the area below the lower horizontal line connecting the third point X3 and the fourth point X4 of the area 720.

[0106] In one embodiment, the electronic device 103 may share (or transfer) (or send) regions 730 and 735 in the image 700 to the electronic device of the following vehicle. In one embodiment, the electronic device 103 may share (or transfer) (or send) a single data obtained by concatenating regions 730 and 735 in the image 700 to the electronic device of the following vehicle. In one embodiment, the electronic device 103 may share (or transfer) (or send) each region 730 and 735 in the image 700 to the electronic device of the following vehicle.

[0107] In one embodiment, the electronic device of the following vehicle can comprehensively identify the driving environment based on areas 730 and 735 in the image 700 obtained from the electronic device of the preceding vehicle and the image obtained by its own camera. For example, the electronic device of the following vehicle can identify the driving environment based on external objects identified by areas 730 and 735 in the image 700 and external objects identified by the image obtained by its own camera.

[0108] According to one embodiment, when the data size of the surrounding environment related information between the electronic devices 101, 103, and 105 is large, the consumption of communication resources may increase. Therefore, a method may be needed to efficiently reduce the size of the data including the surrounding environment related information. Figure 7 , describes operations for reducing data containing information about the surrounding environment.

[0109] According to one embodiment, the image captured by the electronic device 103, 105 mounted on the following vehicle can be shared (or transferred) (or sent) to the electronic device of the preceding vehicle other than the leading vehicle. In one embodiment, the feature map of the image captured by the electronic device 103, 105 mounted on the following vehicle (or the intermediate calculation result of the neural network model 400) (or the output result of the neural network model 400) can be shared (or transferred) (or sent) to the electronic device of the preceding vehicle other than the leading vehicle. For example, the image captured by the electronic device 105 can be shared with the electronic device 103 of the preceding vehicle other than the leading vehicle. For example, the feature map of the image captured by the electronic device 105 (or the output result of the neural network model 400) can be shared (or transferred) (or sent) to the electronic device 103 of the preceding vehicle other than the leading vehicle.

[0110] In one embodiment, the electronic device 103 or 105 mounted on the following vehicle may share (or transmit) a portion of the captured image to the electronic device of the preceding vehicle other than the leading vehicle. In one embodiment, the portion may be an area of ​​the image captured by the electronic device 103 or 105 that does not include the preceding vehicle.

[0111] In one embodiment, the electronic device 103 may share (or transfer) (or send) regions 730 and 735 in the image 700 to the electronic device of the following vehicle. In one embodiment, the electronic device 103 may share (or transfer) (or send) a single data obtained by concatenating regions 730 and 735 in the image 700 to the electronic device of the following vehicle. In one embodiment, the electronic device 103 may share (or transfer) (or send) each region 730 and 735 in the image 700 to the electronic device of the following vehicle.

[0112] In one embodiment, the electronic device of the following vehicle can comprehensively identify the driving environment based on regions 730 and 735 in the image 700 acquired from the electronic device of the preceding vehicle, regions in the image acquired from the camera of the electronic device of the following vehicle, and images acquired from the camera of the electronic device of the following vehicle. For example, the electronic device of the following vehicle can identify the driving environment based on external objects recognized by regions 730 and 735 in the image 700, external objects recognized by regions in the image acquired from the camera of the electronic device of the following vehicle, and external objects recognized by the image acquired from the camera of the electronic device of the following vehicle.

[0113] Figure 8 An example of a method for calculating the width of an object in a front image according to an embodiment is shown.

[0114] In one embodiment, the processor 221 of the electronic device 101 can obtain lane width information (e.g., 3.5 meters) of the road on which the vehicle 210 is traveling via the communication circuit 211. In one embodiment, the lane width information is not obtained through images from the camera 241, but is obtained from a server via the communication circuit 211.

[0115] In one embodiment, the processor 221 may use the camera 241 to identify the lane line of the road on which the vehicle 210 is traveling and its width 835 (or the number of pixels in the width direction). In one embodiment, the processor 221 may use the camera 241 to identify the width 830 (or the number of pixels in the width direction) of the object (e.g., the vehicle 210). For example, the width 835 (or the number of pixels in the width direction) of the lane line and the width 830 (or the number of pixels in the width direction) of the object (e.g., the vehicle 210) may be identified at the same height (or the same y-axis position) within the image 800.

[0116] In one embodiment, the processor 221 may identify the width dimension of the object (e.g., vehicle 210) based on a ratio (e.g., β) between the width 835 (or the number of pixels in the width direction) of the lane line and the width 830 (or the number of pixels in the width direction) of the object (e.g., vehicle 210) and the width information of the lane line (e.g., 3.5 meters). For example, the processor 221 may identify (or calculate) the actual width of the object (e.g., vehicle 210) by multiplying the ratio β by the actual width of the lane line (e.g., 3.5 meters).

[0117] Figure 9 An example of a method for calculating the size of an object according to an embodiment is shown.

[0118] In one embodiment, the processor 221 of the electronic device 101 can identify the height H2 of an object in contact with the ground. For example, the processor 221 can identify the height H2 of the object based on the height (or height from the ground) H1 of the camera 241. For example, the processor 221 can identify the height H2 of the object based on the focal length (or image imaging distance) of the camera 241 (or the distance between the camera 241 and the image). For example, the processor 221 can identify the height of the object based on the height P2 from the bottom of the object (or the ground) to the height H1 of the camera 241 (or the central axis of the camera 241) in the image of the camera 241. For example, the processor 221 can identify the height of the object based on the heights P1 and P2 of the object in the image of the camera 241.

[0119] In one embodiment, the processor 221 can identify (calculate) the actual distance of the object based on the focal length of the camera 241 (or the distance of the image imaging) (or the distance between the camera 241 and the image), the height H1, the height P2 and the trigonometric ratio of the heights P1 and P2 of the object.

[0120] In one embodiment, the processor 221 may identify θ2 based on the focal length of the camera 241 (or the distance at which the image is formed) (or the distance between the camera 241 and the image) and the height P2 from the bottom of the object (or the ground) to the central axis of the camera 241. For example, the processor 221 may identify θ2 based on tan(θ2)=P2 / focal length.

[0121] In one embodiment, the processor 221 may identify θ1 based on the focal length of the camera 241 (or the distance at which the image is formed) (or the distance between the camera 241 and the image) and the height P1 from the central axis of the camera 241 to the top of the object. For example, the processor 221 may identify θ1 based on tan(θ1)=P1 / focal length.

[0122] In one embodiment, the processor 221 may calculate (or identify) the actual distance D between the camera 241 and the object based on the height H1 and θ2 of the camera 241. For example, the processor 221 may identify D based on tan(θ2)=H1 / D.

[0123] In one embodiment, the processor 221 may identify the height H2 of the object in contact with the ground based on the actual distance D, θ1, and θ2 between the camera 241 and the object. For example, the processor 221 may identify H2 based on D×tan(θ1)+D×tan(θ2)=H2.

[0124] As described above, the electronic device 101 can identify the distance D between the camera 241 and the actual object through a monocular camera.

[0125] Figure 10 is an exemplary flow chart of an operating method of an electronic device according to an embodiment.

[0126] Figure 10 You can refer to Figures 1 to 9 Provide explanation. Figure 10 The operations may be performed by the electronic device 101 . Figure 10 The operations may be performed by executing instructions stored in the memory 231 . Figure 10 The operations may be performed by the processor 221 executing instructions stored in the memory 231 .

[0127] Reference Figure 10 In operation 1010, the electronic device 101 may acquire an image 300. According to one embodiment, the electronic device 101 may acquire the image 300 outside the vehicle 210 through the camera 241. The electronic device 101 may use the image 300 acquired by the camera to acquire an image 300 including external objects 270 within a field of view (FoV) of the camera 241.

[0128] In operation 1020, the electronic device 101 may obtain a feature map of the image 300. The electronic device 101 may use the specified neural network model 400 to obtain the feature map of the image 300. For example, the specified neural network model 400 may be composed of a combination of a convolutional layer 410 and a fully connected layer 420. In one embodiment, the feature map may be a calculation result of a layer of the convolutional layer 410.

[0129] In operation 1030, the electronic device 101 may identify the object 270 based on the feature map. The electronic device 101 may input the feature map of the image 300 into the fully connected layer 420 of the neural network model 400 to obtain an object value 452. In one embodiment, the electronic device 101 may classify the type of the object 270 based on the object value 452. In one embodiment, the electronic device 101 may classify the type corresponding to the maximum probability value of the object value 452 among the types that the neural network model 400 can classify as the type of the object 270.

[0130] In operation 1040, the electronic device 101 may send a feature map to the following vehicles (e.g., vehicles 230 and 250). In one embodiment, the electronic device 101 may send the intermediate calculation result (or feature map) of the convolution layer 410 to the electronic devices 103 and 105 via the communication circuit 211. For example, the intermediate calculation result (or feature map) may be the calculation result of a certain layer of the convolution layer 410. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 128×128×128. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 56×56×256. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 28×28×512. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 14×14×512. For example, the intermediate calculation result (or feature map) may be a value represented by a three-dimensional vector of 7×7×512. However, the present invention is not limited thereto.

[0131] In operation 1050 , the electronic device 101 may obtain a determination result from the following vehicles (eg, vehicles 230 and 250 ).

[0132] In one embodiment, electronic devices 103 and 105 can obtain an object value representing the type (or category) of object 270 by inputting the intermediate calculation results into the neural network model 400 stored in their respective memories 233 and 235. In one embodiment, the object value may be the same as or different from the object value 452. In one embodiment, electronic devices 103 and 105 can classify the type of object 270 based on the object value. In one embodiment, electronic devices 103 and 105 can classify the type of object 270 as the type represented by the maximum probability value among the probabilities corresponding to each object value from the types that can be classified by neural network model 400. In one embodiment, the classification results of object 270 by each electronic device 103 and 105 can be the same as or different from the classification result of object 270 by electronic device 101.

[0133] In one embodiment, the electronic devices 103 and 105 may send the object value 452 and / or information related to the type (or category) (or classification result) of the object 270 identified based on the object value 452 to the electronic device 101 through the communication circuits 213 and 215 .

[0134] In operation 1060, the electronic device 101 may determine whether the preceding vehicle needs to be replaced. In one embodiment, the electronic device 101 may determine whether the preceding vehicle needs to be replaced based on the correctness (or accuracy) of the classification results of the object 270. In one embodiment, the correctness (or accuracy) may be calculated based on the correctness and / or errors of the classification results of the object 270 performed by the electronic device 101 while the vehicle 210 is traveling. For example, the correctness (or accuracy) may represent the proportion of correct judgments in the classification results of the objects classified by the electronic device 101.

[0135] In one embodiment, the electronic device 101 may decide to replace the leading vehicle when the accuracy of the electronic device 101 falls below a baseline accuracy (or accuracy) (e.g., 95%). In another embodiment, the electronic device 101 may decide to replace the leading vehicle when the accuracy of the electronic device 101 falls below the accuracy of the electronic devices 103 and 105 in the following vehicles.

[0136] In one embodiment, if it is determined in operation 1060 that the leading vehicle needs to be replaced, the electronic device 101 may perform operation 1070. In one embodiment, if it is determined in operation 1060 that the leading vehicle does not need to be replaced, the electronic device 101 may perform operation 1080.

[0137] In operation 1070 , the electronic device 101 may change the vehicle 210 to the following vehicle. In one embodiment, the electronic device 101 may notify (or send) the electronic devices 103 and 105 of the change of the leading vehicle via the communication circuit 211 based on the change of the leading vehicle.

[0138] In one embodiment, vehicle 210 may temporarily exit the cluster formation due to the lead vehicle being replaced by another vehicle. In one embodiment, vehicle 210 may move to a position behind the lead vehicle. In one embodiment, vehicle 210 may move to the last position in the cluster formation, but this is not limiting.

[0139] In one embodiment, the electronic device 101 may select a new lead vehicle based on the accuracy of the electronic devices 103 and 105 included in the following vehicles. For example, the electronic device 101 may select the vehicle equipped with the electronic device with the highest accuracy as the new lead vehicle. In one embodiment, based on the selection (or determination) of the new lead vehicle, the processor 221 may notify (or transmit) information about the new lead vehicle to the electronic devices 103 and 105 via the communication circuit 211.

[0140] In operation 1080 , the electronic device 101 may maintain the vehicle 210 as a leading vehicle.

[0141] Figure 11 is an exemplary flow chart of an operating method of an electronic device according to an embodiment.

[0142] Figure 11 You can refer to Figures 1 to 9 Provide explanation. Figure 11 The operations may be performed by the electronic devices 103 and 105 . Figure 11 The operations can be performed by executing instructions stored in the memories 233 and 235. Figure 11 The operations can be performed by the processors 223 and 225 executing the instructions stored in the memories 233 and 235. Figure 11 The operations may be exemplified as being performed by the electronic device 105 .

[0143] Reference Figure 11 In operation 1110, the electronic device 105 may acquire an image 600. According to one embodiment, the electronic device 105 may acquire an image 600 outside the vehicle 250 through the camera 245. The electronic device 105 may use the image 600 acquired through the camera to acquire an image 600 including an external object 690 within a field of view (FoV) of the camera 245.

[0144] In operation 1120, the electronic device 105 may recognize the object 690 based on the image 600. According to one embodiment, the electronic device 105 may use a specified neural network model (e.g., Figure 4 ), identifying an object 690 and a type (or category) of the object 690 contained in the image 600. In one embodiment, the electronic device 105 may identify an object of a specified type (e.g., a vehicle, a traffic sign, or a road marking) from the image 600 using an image segmentation model and / or an object detection model.

[0145] In one embodiment, the electronic device 105 may classify the type of the object 690 based on the object value in the image 600 recognized by the neural network model 400. In one embodiment, the electronic device 105 may classify the type represented by the maximum probability value among the probabilities corresponding to the object value from the types that the neural network model 400 can classify as the type of the object 690. However, this is not limiting. In one embodiment, the electronic device 105 may only set a specific portion (or region of interest (ROI)) of the image 600 as the input data of the neural network model 400.

[0146] In operation 1130, the electronic device 105 may identify a priority based on the object 690. In one embodiment, the electronic device 105 may determine the priority between the electronic devices 101 and 103 based on the risk level of the external object 690. For example, among the electronic devices 101 and 103, the electronic device with the highest risk level caused by the external object 690 may have the highest priority. For example, among the electronic devices 101 and 103, the electronic device with the lowest risk level caused by the external object 690 may have the lowest priority. For example, the risk level of the external object 690 may be a possibility of collision with the electronic device due to the position, movement direction and / or movement speed of the external object 690. For example, the higher the possibility of collision, the higher the risk level may be assessed to be.

[0147] In operation 1140, the electronic device 105 may send a notification based on priority. In one embodiment, the electronic device 105 may send data containing information related to the surrounding environment to the electronic devices 101 and 103 based on the priority of the electronic devices 101 and 103. In one embodiment, the electronic device 105 may send data containing information related to the surrounding environment to the electronic devices 101 and 103 in sequence according to the order of priority. For example, the electronic device 105 may preferentially send data containing information related to the surrounding environment to the electronic device 103 contained in the vehicle 230 with a high possibility of collision caused by the object 690. Subsequently, the electronic device 105 may send data containing information related to the surrounding environment to the electronic device 101 contained in the vehicle 210 with a relatively low possibility of collision caused by the object 690.

[0148] Figure 12 An example block diagram of an autonomous driving system for a vehicle according to an embodiment is shown.

[0149] according to Figure 12, the vehicle automatic driving system 1200 can be a deep learning network including a sensor 1203, an image preprocessor 1205, a deep learning network 1207, an artificial intelligence (AI) processor 1209, a vehicle control module 1211, a network interface 1213 and a communication unit 1215. In various embodiments, the various components can be connected through different interfaces. For example, the sensor data sensed and output by the sensor 1203 can be fed to the image preprocessor 1205. The sensor data processed by the image preprocessor 1205 can be fed to the deep learning network 1207 run by the AI ​​processor 1209. The output of the deep learning network 1207 run by the AI ​​processor 1209 can be fed to the vehicle control module 1211. The intermediate results of the deep learning network 1207 running on the AI ​​processor 1209 can be fed to the AI ​​processor 1209. In various embodiments, the network interface 1213 can be connected to the in-vehicle electronic devices (e.g., Figure 2 The electronic device 101 and / or the electronic devices 103 and 105 in the autonomous driving control system 1200 communicate with each other to pass the autonomous driving path information and / or autonomous driving control instructions for the autonomous driving of the vehicle to the internal module. In one embodiment, the network interface 1213 can be used to transmit the sensor data obtained by the sensor 1203 to an external server. In some embodiments, the autonomous driving control system 1200 may include additional or fewer components as appropriate. For example, in some embodiments, the image preprocessor 1205 may be an optional component. For another example, a post-processing component (not shown) may be included in the autonomous driving control system 1200 to perform post-processing on the output of the deep learning network 1207 before providing the output to the vehicle control module 1211.

[0150] In some embodiments, sensor 1203 may include more than one sensor. In various embodiments, sensor 1203 may be installed at different locations on the vehicle. Sensor 1203 may face one or more different directions. For example, sensor 1203 may be installed on the front, sides, rear, and / or roof of the vehicle, facing forward, rear, or sideways, among other directions. In some embodiments, sensor 1203 may be an image sensor, such as a high dynamic range camera. In some embodiments, sensor 1203 may include non-visual sensors. In some embodiments, sensor 1203 may include radar, laser radar (LiDAR), and / or ultrasonic sensors in addition to image sensors. In some embodiments, sensor 1203 is not mounted on the vehicle having vehicle control module 1211. For example, sensor 1203 may be part of a deep learning system to capture sensor data and may be installed in the environment or on a road, and / or installed on surrounding vehicles.

[0151] In some embodiments, the image pre-processor 1205 can be used to pre-process the sensor data of the sensor 1203. For example, the image pre-processor 1205 can be used to pre-process the sensor data, split the sensor data into one or more constituent elements, and / or post-process one or more constituent elements. In some embodiments, the image pre-processor 1205 can be a graphics processing unit (GPU), a central processing unit (CPU), an image signal processor, or a specialized image processor. In various embodiments, the image pre-processor 1205 can be a tone-mapper processor for processing high dynamic range data. In some embodiments, the image pre-processor 1205 can be a component of the AI ​​processor 1209.

[0152] In some embodiments, deep learning network 1207 may be a deep learning network for implementing control commands for controlling an autonomous vehicle. For example, deep learning network 1207 may be an artificial neural network, such as a convolutional neural network (CNN), trained using sensor data, and the output of deep learning network 1207 is provided to vehicle control module 1211.

[0153] In some embodiments, the artificial intelligence (AI) processor 1209 may be a hardware processor for running the deep learning network 1207. In some embodiments, the AI ​​processor 1209 may be a specialized AI processor for performing inference on sensor data using a convolutional neural network (CNN). In some embodiments, the AI ​​processor 1209 may be optimized for the bit depth of the sensor data. In some embodiments, the AI ​​processor 1209 may be optimized for deep learning operations (such as operations in a neural network that include convolution, inner product, vector and / or matrix operations). In some embodiments, the AI ​​processor 1209 may be implemented using multiple graphics processing units (GPUs) that can efficiently perform parallel processing.

[0154] In various embodiments, the AI ​​processor 1209 can be coupled to a memory storing instructions via an input / output interface. When executed by the AI ​​processor 1209, the instructions can perform deep learning analysis on sensor data from the sensor 1203 and generate machine learning results for enabling at least partially autonomous operation of the vehicle. In certain embodiments, the vehicle control module 1211 can process vehicle control instructions output by the artificial intelligence (AI) processor 1209 and translate the output of the AI ​​processor 1209 into instructions for controlling various modules of the vehicle. In certain embodiments, the vehicle control module 1211 can be used to control the vehicle to achieve autonomous driving. In certain embodiments, the vehicle control module 1211 can adjust the steering and / or speed of the vehicle. For example, the vehicle control module 1211 can be used to control the vehicle's driving, including operations such as deceleration, acceleration, steering, lane changing, and lane keeping. In some embodiments, the vehicle control module 1211 can generate control signals for controlling vehicle lighting, such as brake lights, turn signals, and headlights. In some embodiments, the vehicle control module 1211 can be used to control vehicle audio-related systems, such as the vehicle's sound system, the vehicle's audio warnings, the vehicle's microphone system, and the vehicle's horn system.

[0155] In some embodiments, the vehicle control module 1211 can be used to control notification systems, including warning systems for alerting passengers and / or drivers to driving events, such as approaching a predetermined destination or a potential collision. In some embodiments, the vehicle control module 1211 can be used to adjust vehicle sensors, such as sensor 1203. For example, the vehicle control module 1211 can modify the orientation of sensor 1203, change the output resolution and / or format type of sensor 1203, increase or decrease the capture rate, adjust the dynamic range, and adjust the focus of the camera. In addition, the vehicle control module 1211 can turn the operation of sensors on or off individually or collectively.

[0156] In certain embodiments, the vehicle control module 1211 can be used to modify parameters of the image preprocessor 1205, such as adjusting the frequency range of the filter, adjusting edge detection parameters for feature and / or object detection, or adjusting channels and bit depth. In various embodiments, the vehicle control module 1211 can be used to control autonomous driving functions and / or driver assistance functions of the vehicle.

[0157] In certain embodiments, the network interface 1213 may serve as an internal interface between modules of the autonomous driving control system 1200 and the communication unit 1215. Specifically, the network interface 1213 may serve as a communication interface for receiving and / or transmitting data, including voice data. In various embodiments, the network interface 1213 may connect to an external server via the communication unit 1215 to facilitate voice call connections, receive and / or transmit text messages, transmit sensor data, and update the vehicle's software to the autonomous driving system or the vehicle's autonomous driving system software.

[0158] In various embodiments, the communication unit 1215 may include a variety of wireless interfaces such as cellular or WiFi. For example, the network interface 1213 may be connected to an external server via the communication unit 1215 to receive updates on operating parameters and / or instructions for the sensor 1203, the image preprocessor 1205, the deep learning network 1207, the AI ​​processor 1209, and the vehicle control module 1211. For example, the machine learning model of the deep learning network 1207 may be updated via the communication unit 1215. In another example, the communication unit 1215 may be used to update operating parameters (such as image processing parameters) of the image preprocessor 1205 and / or the firmware of the sensor 1203.

[0159] In other embodiments, the communication unit 1215 can be used to activate communications with emergency services and emergency contacts in the event of an accident or near-accident. For example, in the event of a collision, the communication unit 1215 can be used to call emergency services for assistance and to notify emergency services of details of the collision and the location of the vehicle. In various embodiments, the communication unit 1215 can also be used to update or obtain an estimated time of arrival and / or destination location.

[0160] According to one embodiment, Figure 12The illustrated autonomous driving system 1200 may be comprised of the vehicle's electronic device 101. According to one embodiment, when a user triggers an autonomous driving release event during the vehicle's autonomous driving process, the AI ​​processor 1209 of the autonomous driving system 1200 may control the vehicle's autonomous driving software to learn by inputting information related to the autonomous driving release event into the training set data of a deep learning network.

[0161] Figure 13 and Figure 14 An example block diagram of an autonomous driving mobile body according to an embodiment is shown. Figure 15 An example of a gateway is shown in relation to a user device in various embodiments.

[0162] Reference Figure 13 According to this embodiment, the autonomous driving mobile body 1300 may include a control device 1400, perception modules 1304a, 1304b, 1304c, 1304d, an engine 1306 and a user interface 1308.

[0163] The autonomous vehicle 1300 may have an autonomous driving mode or a manual mode. For example, the vehicle may be switched from the manual mode to the autonomous driving mode or vice versa based on user input received through the user interface 1308.

[0164] When the moving object 1300 operates in the autonomous driving mode, the autonomous driving moving object 1300 may operate under the control of the control device 1400 .

[0165] In this embodiment, the control device 1400 may include a controller 1420 having a memory 1422 and a processor 1424 , a sensor 1410 , a communication device 1430 , and an object detection device 1440 .

[0166] The object detection device 1440 may perform all or part of the functions of the distance measurement device.

[0167] That is, in this embodiment, the object detection device 1440 is a device for detecting an object located outside the moving body 1300. The object detection device 1440 can detect an object located outside the moving body 1300 and generate object information based on the detection result.

[0168] The object information may include information on the presence or absence of the object, position information of the object, distance information between the moving body and the object, and relative speed information between the moving body and the object.

[0169] Objects may include lane markings, other vehicles, pedestrians, traffic signals, light, roads, structures, speed bumps, terrain features, animals, and other objects located outside of the mobile object 1300. Traffic signals may include traffic lights, traffic signs, and patterns or text painted on the road surface. Furthermore, light may be generated by lights equipped by other vehicles, light generated by streetlights, or sunlight.

[0170] Furthermore, structures can be objects located around the road and fixed to the ground. For example, structures can include streetlights, roadside trees, buildings, utility poles, traffic lights, and bridges. Terrain objects can include mountains and hills.

[0171] The object detection device 1440 may include a camera module. The controller 1420 may extract object information from an external image captured by the camera module and process the information related thereto.

[0172] Furthermore, object detection device 1440 may also include an imaging device for sensing the external environment. In addition to LIDAR, RADAR, GPS devices, odometry and other computer vision devices, ultrasonic sensors, and infrared sensors may also be used. These devices can be used selectively or simultaneously as needed to achieve more accurate detection.

[0173] On the other hand, according to an embodiment of the present invention, the distance measuring device can calculate the distance between the autonomous driving mobile body 1300 and the object, and in conjunction with the control device 1400 of the autonomous driving mobile body 1300, control the movement of the mobile body based on the calculated distance.

[0174] For example, when the distance between the autonomous vehicle 1300 and an object is likely to conflict, the autonomous vehicle 1300 can control the brakes to reduce speed or stop. Another example is when the object is moving, the autonomous vehicle 1300 can control its speed to maintain a predetermined distance from the object.

[0175] According to an embodiment of the present invention, such a distance measurement device may be configured as a module in the control device 1400 of the autonomous driving mobile body 1300. In other words, the memory 1422 and processor 1424 of the control device 1400 may implement the anti-collision method of the present invention in software.

[0176] In addition, the sensor 1410 can be connected to the sensing modules 1304a, 1304b, 1304c, and 1304d to obtain various sensing information of the internal / external environment of the mobile object. The sensor 1410 may include a posture sensor (e.g., a yaw sensor, a roll sensor, a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, a tilt sensor, a weight detection sensor, a heading sensor, a gyro sensor, a position module, a mobile forward / backward sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor detected by steering wheel rotation, a mobile internal temperature sensor, a mobile internal humidity sensor, an ultrasonic sensor, a light sensor, an accelerator pedal position sensor, a brake pedal position sensor, and the like.

[0177] Therefore, the sensor 1410 can obtain perception signals about the following: mobile body posture information, mobile body collision information, mobile body direction information, mobile body position information (GPS information), mobile body angle information, mobile body speed information, mobile body acceleration information, mobile body tilt information, mobile body forward / backward information, battery information, fuel information, tire information, mobile body light information, mobile body internal temperature information, mobile body internal humidity information, steering wheel rotation angle, mobile body external lighting, accelerator pedal pressure, and brake pedal pressure, etc.

[0178] In addition, sensor 1410 may also include other sensors, such as an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), a throttle position sensor (TPS), a TDC sensor, and a crankshaft angle sensor (CAS).

[0179] As described above, the sensor 1410 may generate mobile object state information based on the sensing data.

[0180] The wireless communication device 1430 is configured to implement wireless communication between the autonomous driving mobile bodies 1300. For example, the autonomous driving mobile body 1300 can communicate with a user's mobile phone, other wireless communication devices 1430, other mobile bodies, a central device (such as a traffic control device), a server, etc. The wireless communication device 1430 can send and receive wireless signals according to the access wireless protocol. The wireless communication protocol may include Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), and Global Systems for Mobile Communications (GSM), but is not limited to these protocols.

[0181] In addition, according to this embodiment, the autonomous driving mobile body 1300 can also achieve communication between mobile bodies through the wireless communication device 1430. That is, the wireless communication device 1430 can communicate with other mobile bodies and other vehicles on the road through vehicle-to-vehicle (V2V) communication. The autonomous driving mobile body 1300 can send and receive data such as driving warnings and traffic information through inter-vehicle communication, and can also request information from other mobile bodies or receive requests from other mobile bodies. For example, the wireless communication device 1430 can achieve V2V communication through dedicated short-range communication (DSRC) equipment or C-V2V (Cellular-V2V) equipment. In addition to inter-vehicle communication, communication between the vehicle and other things (such as electronic devices carried by pedestrians, etc.) (V2X, Vehicle-to-Everything communication) can also be achieved through the wireless communication device 1430.

[0182] In addition, the wireless communication device 1430 can also obtain information from various mobile bodies (Mobility) such as infrastructure on the road (such as traffic lights, CCTV, RSU, eNode B, etc.) or other autonomous driving (Autonomous Driving) / non-autonomous driving (Non-AutonomousDriving) vehicles through a non-terrestrial network (Non-Terrestrial Network), and use it as the information required for the autonomous driving mobile body 1300 to perform autonomous driving.

[0183] For example, the wireless communication device 1430 can wirelessly communicate with the low Earth orbit (LEO) satellite system, medium Earth orbit (MEO) satellite system, geostationary orbit (GEO) satellite system, high altitude platform (HAP) system, etc. that constitute the non-terrestrial network through a non-terrestrial network dedicated antenna mounted on the autonomous driving mobile body 1300.

[0184] For example, the wireless communication device 1430 can conduct wireless communications with various platforms constituting the NTN based on the wireless access specifications of the 5G NR NTN (5th Generation New Radio Non-Terrestrial Network) standard specifications currently under discussion by organizations such as 3GPP, but is not limited thereto.

[0185] In this embodiment, the controller 1420 can consider various information such as the location, current time, available power, etc. of the autonomous driving mobile body 1300, select a platform that can appropriately perform NTN communication, and control the wireless communication device 1430 to perform wireless communication with the selected platform.

[0186] In this embodiment, controller 1420, which controls the overall operation of various units within mobile object 1300, can be configured by the mobile object's manufacturer during manufacturing or further configured after manufacturing to implement autonomous driving functions. Alternatively, it can include a configuration that continuously executes additional functions by upgrading the controller 1420 configured at manufacturing time. This type of controller 1420 may also be referred to as an ECU (Electronic Control Unit).

[0187] Controller 1420 can collect various data from connected sensors 1410, object detection device 1440, communication device 1430, etc., and based on the collected data, transmit control signals to other components of the mobile body, including sensors 1410, engine 1306, user interface 1308, communication device 1430, and object detection device 1440. In addition, although not specifically described, control signals can also be transmitted to an acceleration device, braking system, steering device, or navigation device related to the movement of the mobile body.

[0188] In this embodiment, the controller 1420 can control the engine 1306. For example, when the autonomous vehicle 1300 detects a speed limit on the road, the controller 1420 can control the engine 1306 to ensure that the driving speed does not exceed the speed limit, or to accelerate the driving speed of the autonomous vehicle 1300 within a range that does not exceed the speed limit.

[0189] Furthermore, when the autonomous vehicle 1300 approaches or deviates from a lane line during its travel, the controller 1420 can determine whether such approach or deviation constitutes a normal driving situation or another driving situation, and control the engine 1306 based on the determination to adjust the vehicle's travel. Specifically, the autonomous vehicle 1300 can detect lane lines formed on both sides of the lane in which the vehicle is traveling. In this case, the controller 1420 can determine whether the autonomous vehicle 1300 is approaching or deviating from a lane line. If it is determined that the autonomous vehicle 1300 is approaching or deviating from a lane line, the controller 1420 can further determine whether such movement is due to a normal driving situation or another driving situation. Here, as an example of a normal driving situation, the vehicle may need to change lanes. Alternatively, as an example of another driving situation, the vehicle may not need to change lanes. If the controller 1420 determines that the autonomous vehicle 1300 is approaching or deviating from a lane line when a lane change is not necessary, the controller 1420 can control the autonomous vehicle 1300 to travel normally in the relevant lane without deviating from the lane line.

[0190] When there are other moving objects or obstacles ahead of the moving object, the engine 1306 or the braking system can be controlled to decelerate the moving object. In addition to speed, the trajectory, driving path, and steering angle can also be controlled. Alternatively, the controller 1420 can generate the necessary control signals to control the moving object based on information about the moving object's lane, driving signals, and other external environments.

[0191] In addition to generating its own control signals, the controller 1420 can also communicate with surrounding mobile objects or a central server, and send commands to control surrounding devices through the received information, thereby controlling the travel of the mobile object.

[0192] Furthermore, if the position or viewing angle of the camera module 1450 changes, the controller 1420 may have difficulty accurately identifying the moving object or lane marking as in the present embodiment. To prevent this, the controller 1420 may also generate a control signal to calibrate the camera module 1450. Therefore, in this embodiment, the controller 1420 issues a calibration control signal to the camera module 1450. This ensures that the camera module 1450 maintains its normal installation position, orientation, and viewing angle, even if the installation position of the camera module 1450 changes due to vibration or impact generated by the movement of the autonomous driving moving object 1300. The controller 1420 may generate a control signal to calibrate the camera module 1420 when a change exceeds a threshold between the pre-stored initial installation position, orientation, and viewing angle of the camera module 1420 and the initial installation position, orientation, and viewing angle of the camera module 1420 measured during driving of the autonomous driving moving object 1300.

[0193] In this embodiment, the controller 1420 may include a memory 1422 and a processor 1424. The processor 1424 may execute software stored in the memory 1422 in response to control signals from the controller 1420. Specifically, the controller 1420 stores data and commands required for executing the lane detection method described in the present invention in the memory 1422. These commands may be executed by the processor 1424 to implement one or more methods disclosed herein.

[0194] In this case, the memory 1422 may be stored on a recording medium executable by the non-volatile processor 1424. The memory 1422 may store software and data via appropriate internal or external devices. The memory 1422 may be composed of RAM (random access memory), ROM (read only memory), a hard disk, or a memory device connected to a dongle.

[0195] The memory 1422 can store at least an operating system (OS), user applications, and executable commands. The memory 1422 can also store application data and array data structures.

[0196] Processor 1424 may be a microprocessor or suitable electronic processor, and may be a controller, microcontroller, or state machine.

[0197] The processor 1424 may be implemented by a combination of computing devices, which may be composed of a digital signal processor, a microprocessor, or a suitable combination thereof.

[0198] On the other hand, the autonomous mobile object 1300 may also include a user interface 1308 for receiving user input to the control device 1400. The user interface 1308 allows the user to input information through an appropriate interactive method. For example, this may be implemented through a touch screen, keyboard, or operation buttons. The user interface 1308 transmits the input or command to the controller 1420, which then executes control actions for the mobile object in response to the input or command.

[0199] In addition, the user interface 1308 can also communicate with devices outside the autonomous driving mobile body 1300 through the wireless communication device 1430. For example, the user interface 1308 can be linked with a mobile phone, tablet computer, or other computer device.

[0200] Furthermore, while the autonomously driven vehicle 1300 described in this embodiment includes an engine 1306, it may also include other types of propulsion systems. For example, the vehicle may be operated by electricity, hydrogen, or a hybrid system comprising a combination thereof. Therefore, the controller 1420 includes a propulsion mechanism specific to the propulsion system of the autonomously driven vehicle 1300 and may provide corresponding control signals to the components of each propulsion mechanism.

[0201] Below, refer to Figure 14 , further describing in detail the detailed structure of the control device 1400 according to this embodiment.

[0202] The control device 1400 includes a processor 1424. The processor 1424 can be a general-purpose single-chip or multi-chip microprocessor, a dedicated microprocessor, a microcontroller, a programmable gate array, etc. The processor can also be referred to as a central processing unit (CPU). In addition, in this embodiment, the processor 1424 can also be used as a combination of multiple processors.

[0203] The control device 1400 further includes a memory 1422. The memory 1422 may be any electronic component capable of storing electronic information. In addition to being a single memory, the memory 1422 may also include a combination of multiple memories 1422.

[0204] The data and command 1422a required for the distance measurement device according to the present invention to execute the distance measurement method may be stored in the memory 1422. When the processor 1424 executes the command 1422a, all or part of the command 1422a and the data 1422b required to execute the command may be loaded onto the processor 1424 (1424a, 1424b).

[0205] The control device 1400 may include a transmitter 1430a, a receiver 1430b, or a transceiver 1430c for allowing signal transmission and reception. One or more antennas 1432a, 1432b may be electrically connected to the transmitter 1430a, the receiver 1430b, or each transceiver 1430c, and may further include an antenna.

[0206] The control device 1400 may further include a digital signal processor (DSP) 1470. The DSP 1470 allows the mobile device to quickly process digital signals.

[0207] The control device 1400 may further include a communication interface 1480. The communication interface 1480 may include one or more ports and / or communication modules for connecting other devices to the control device 1400. The communication interface 1480 allows a user to interact with the control device 1400.

[0208] The various components of the control device 1400 can be connected via one or more buses 1490, and the buses 1490 can include a power bus, a control signal bus, a status signal bus, a data bus, etc. Under the control of the processor 1424, the components can communicate information with each other via the buses 1490 and perform predetermined functions.

[0209] On the other hand, in various embodiments, the control device 1400 can be associated with a gateway to communicate with the secure cloud. Figure 15 , the control device 1400 may be associated with a gateway 1505 for providing information acquired from at least one of the components (1501 to 1504) of the vehicle 1500 to a secure cloud 1506. For example, the gateway 1505 may be included in the control device 1400. As another example, the gateway 1505 may be configured as an independent device within the vehicle 1500, separate from the control device 1400. The gateway 1505 can connect the software management cloud 1509 and the secure cloud 1506, which have different networks, and the internal network of the vehicle 1500 protected by the in-vehicle security software 1510, to achieve communication.

[0210] For example, component 1501 may be a sensor. For example, the sensor may be used to obtain information about at least one of the state of vehicle 1500 or the state around vehicle 1500. For example, component 1501 may include sensor 1410.

[0211] For example, component 1502 may be an ECU (electronic control unit), which may be used for engine control, transmission control, airbag control, or tire pressure management.

[0212] For example, component 1503 may be an instrument cluster. For example, the instrument cluster may be a panel located in front of the driver's seat in a dashboard. For example, the instrument cluster may be configured to display information necessary for driving to the driver (or passenger). For example, the instrument cluster may display at least one of a visual element indicating engine revolutions per minute (RPM), a visual element indicating the speed of vehicle 1500, a visual element indicating the remaining fuel level, a visual element indicating the gear status, and a visual element indicating information obtained through component 1501.

[0213] For example, component 1504 may be a telematics device. For example, the telematics device may be a device that provides various mobile communication services such as location information and safe driving in the vehicle 1500 by combining wireless communication technology and GPS (global positioning system) technology. For example, the telematics device may be used to connect the vehicle 1500 to the driver, the cloud (e.g., the safety cloud 1506) and / or the surrounding environment. For example, the telematics device may support high bandwidth and low latency for 5G NR specification technology (e.g., 5G NR's V2X technology, 5G NR's NTN (Non-Terrestrial Network) technology). For example, the telematics device may support autonomous driving of the vehicle 1500.

[0214] For example, gateway 1505 can be used to connect the network within vehicle 1500 with a software management cloud 1509 and a security cloud 1506, which are external networks. For example, software management cloud 1509 can be used to update or manage at least one software required for driving and managing vehicle 1500. For example, software management cloud 1509 can be linked with in-car security software 1510 installed within the vehicle. For example, in-car security software 1510 can be used to provide security functions within vehicle 1500. For example, in-car security software 1510 can encrypt data sent and received via the in-car network using an encryption key obtained from an external authorized server to encrypt the in-vehicle network. In various embodiments, the encryption key used by in-vehicle security software 1510 can be generated in response to vehicle identification information (vehicle license plate, vehicle identification number (VIN)) or information uniquely assigned to each user (e.g., user identification information).

[0215] In various embodiments, the gateway 1505 can transmit data encrypted by the in-vehicle security software 1510 using the encryption key to the software management cloud 1509 and / or the security cloud 1506. The software management cloud 1509 and / or the security cloud 1506 decrypt the data encrypted by the in-vehicle security software 1510 using the encryption key using a decryption key capable of decrypting the data, thereby identifying the vehicle or user from which the data was received. For example, because the decryption key is a unique key corresponding to the encryption key, the software management cloud 1509 and / or the security cloud 1506 can identify the data sender (e.g., the vehicle or user) based on the data decrypted using the decryption key.

[0216] For example, gateway 1505 is configured to support in-vehicle security software 1510 and may be associated with control device 1400. For example, gateway 1505 may be associated with control device 1500 to support a connection between control device 1500 and client device 1507 connected to secure cloud 1506. According to another example, gateway 1505 may be associated with control device 1400 to support a connection between control device 1400 and a third-party cloud 1508 connected to secure cloud 1506. However, the present invention is not limited thereto.

[0217] In various embodiments, gateway 1505 can be used to connect vehicle 1500 with a software management cloud 1509 for managing the operating software of vehicle 1500. For example, software management cloud 1509 monitors whether the operating software of vehicle 1500 needs to be updated and, upon detecting that the operating software of vehicle 1500 needs to be updated, provides data for updating the operating software of vehicle 1500 via gateway 1505. In another example, software management cloud 1509 receives a user request for updating the operating software of vehicle 1500 from vehicle 1500 via gateway 1505 and, based on the request, provides data for updating the operating software of vehicle 1500. However, the present invention is not limited to this.

[0218] Figure 16 is a diagram for explaining operation of an electronic device for training a neural network based on a training data set according to an embodiment.

[0219] Reference Figure 16 The described operations can be performed by the above-mentioned electronic devices (e.g., Figure 2 Executed by the electronic device 101).

[0220] refer to Figure 16In operation 1602, according to one embodiment, the electronic device may obtain a training data set. The electronic device may obtain a training data set for supervised learning. The training data may include input data and ground truth data pairs corresponding to the input data. The ground truth data may represent output data to be obtained from a neural network that receives input data of the ground truth data pair. The ground truth data may be obtained by the aforementioned electronic device.

[0221] For example, when training a neural network to recognize an image, the training data may include information about the image and one or more objects contained in the image. The information may include a category (category or class) of an object that can be recognized by the image. The information may include the position, width, height and / or size of a visual object corresponding to the object in the image. The training data set identified by operation 1602 may include multiple training data pairs. In the example of training a neural network to recognize an image, the training data set identified by the electronic device may include multiple images and ground truth data corresponding to each of the multiple images.

[0222] Reference Figure 16 In operation 1604, according to one embodiment, the electronic device may train the neural network based on the training data set. In one embodiment of training the neural network based on supervised learning, the electronic device may input the input data contained in the training data into the input layer of the neural network. Figure 17 An example of a neural network including the input layer is described. The electronic device can obtain output data of the neural network corresponding to the input data from the output layer of the neural network that receives the input data through the input layer.

[0223] In one embodiment, the training of operation 1604 may be performed based on the difference between the output data and the ground truth data corresponding to the input data included in the training data. For example, the electronic device may adjust one or more parameters related to the neural network (e.g., the reference Figure 17 The electronic device may adjust the one or more parameters to reduce the difference. The operation of the electronic device to adjust the one or more parameters may be referred to as tuning the neural network. The electronic device may perform neural network tuning based on the output data by using a function defined for evaluating the performance of the neural network, such as a cost function. The difference between the output data and the ground truth data may be included in an example of the cost function.

[0224] refer to Figure 16 In operation 1606, according to one embodiment, the electronic device may identify whether the neural network trained in operation 1604 outputs valid output data. Valid output data means that the difference (or cost function) between the output data and the ground truth data satisfies the conditions set for using the neural network. For example, when the average value and / or the maximum value of the difference between the output data and the ground truth data is less than or equal to a specified threshold, the electronic device may determine that the neural network has output valid output data.

[0225] If the neural network does not output valid output data (1606-No), the electronic device may repeatedly perform training of the neural network based on operation 1604. The embodiment is not limited thereto, and the electronic device may repeatedly perform operations 1602 and 1604.

[0226] When valid output data is obtained from the neural network (1606-Yes), the electronic device according to one embodiment may use the trained neural network based on operation 1608. For example, the electronic device may input other input data, which is different from the input data input to the neural network as training data, into the neural network. The electronic device may use the output data obtained from the neural network that received the other input data as the result of inference performed on the other input data by the neural network.

[0227] Figure 17 is a block diagram of an electronic device according to an embodiment.

[0228] Figure 17 The electronic device 101 may include the aforementioned electronic devices.

[0229] For example, refer to Figure 17 The described operation can be performed by Figure 17 The electronic device 101 and / or Figure 17 Executed by processor 1710.

[0230] Reference Figure 17, the processor 1710 of the electronic device 101 can perform computations related to the neural network 1730 stored in the memory 1720. The processor 1710 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), or a neural processing unit (NPU). The NPU can be implemented as a chip separate from the CPU, or integrated in a chip such as a CPU in the form of a system on a chip (SoC). The NPU integrated in the CPU can be called a neural core and / or an artificial intelligence (AI) accelerator.

[0231] Reference Figure 17 , the processor 1710 may identify a neural network 1730 stored in the memory 1720. The neural network 1730 may include a combination of an input layer 1732, one or more hidden layers 1734 (or intermediate layers), and an output layer 1736. Each of the above layers (e.g., the input layer 1732, one or more hidden layers 1734, and the output layer 1736) may include multiple nodes. The number of hidden layers 1734 may vary depending on the embodiment, and a neural network 1730 including multiple hidden layers 1734 may be referred to as a deep neural network. The operation of training the deep neural network may be referred to as deep learning.

[0232] In one embodiment, if neural network 1730 has a feedforward neural network structure, a first node included in a particular layer may be connected to all second nodes included in other layers preceding the particular layer. Parameters stored for neural network 1730 in memory 1720 may include weights assigned to connections between the second nodes and the first node. In neural network 1730 having a feedforward neural network structure, the value of the first node may correspond to a weighted sum of values ​​assigned to the second node based on the weights assigned to the connections connecting the second node and the first node.

[0233] In one embodiment, if neural network 1730 has a convolutional neural network structure, a first node included in a specific layer may correspond to a weighted sum of some second nodes included in other layers before the specific layer. The part of the second nodes corresponding to the first node may be identified by a filter corresponding to the specific layer. Parameters stored in memory 1720 for neural network 1730 may include weights representing the filter. The filter may include one or more nodes in the second node for calculating the weighted sum of the first node, and weights corresponding to each of the one or more nodes.

[0234] According to one embodiment, the processor 1710 of the electronic device 101 can train the neural network 1730 using the training data set 1740 stored in the memory 1720. Based on the training data set 1740, the processor 1710 can execute a reference Figure 17 The described operations thereby adjust one or more parameters stored in memory 1720 for neural network 1730.

[0235] According to one embodiment, the processor 1710 of the electronic device 101 can perform object detection, object recognition, and / or object classification using a neural network 1730 trained based on a training data set 1740. The processor 1710 can input an image (or video) acquired through the camera 1750 into the input layer 1732 of the neural network 1730. Based on the input layer 1732 into which the image is input, the processor 1710 can sequentially acquire the values ​​of the nodes of each layer included in the neural network 1730, thereby acquiring a set of values ​​(e.g., output data) of the nodes of the output layer 1736. The output data can be used as the result of reasoning about the information contained in the image using the neural network 1730. The embodiment is not limited thereto, and the processor 1710 can also acquire an image (or video) from an external electronic device connected to the electronic device 101 via the communication circuit 1760 and input it into the neural network 1730.

[0236] In one embodiment, the neural network 1730 trained to process an image can be used to identify regions corresponding to objects in the image (object detection) and / or identify the categories of objects represented in the image (object recognition and / or object classification). For example, the electronic device 101 can use the neural network 1730 to segment regions corresponding to the object in the image based on rectangular shapes, such as bounding boxes. For example, the electronic device 101 can use the neural network 1730 to identify at least one category matching the object from a plurality of specified categories.

[0237] A conventional truck 10 is shown in FIG. Figure 18 and Figure 19 .

[0238] Figure 18 The tractor 12 is shown in a state where it is not connected to the trailer 14 .

[0239] Figure 19 Shown is a state where the tractor 12 is connected to a trailer 14. In an embodiment of the present invention, the trailer 14 can be selectively connected via a steerable wheel hook 16 provided on the tractor 12, which is connected to a traction pin 18 fixed to the trailer 14 in a known manner.

[0240] This manual Figure 18 Although the trailer 20 is shown in a "semi-trailer" form, this is only for the convenience of explanation and it should not be understood that the embodiments of the present invention are only applicable to the "semi-trailer" form.

[0241] The electronic device 101 described above may include: a camera 241; a communication circuit 211; a memory 231 storing a neural network model 400 and instructions; and a processor 221 operatively connected to the camera 241, the communication circuit 211, and the memory 231. The instructions, when executed by the processor 221, enable the electronic device 101 to acquire an image 300 via the camera 241. The instructions, when executed by the processor 221, enable the electronic device 101 to use the image 300 and the neural network model 400 to identify a first classification result of an object 270 contained in the image 300. The instructions, when executed by the processor 221, enable the electronic device 101 to transmit, via the communication circuit 211, a feature map of the neural network model 400 based on the image 300 to an external electronic device 103 or 105 contained in a subsequent vehicle 230 or 250 in which the electronic device 101 is carried. When executed by the processor 221, the instructions enable the electronic device 101 to obtain, from the external electronic devices 103 and 105 via the communication circuit 211, a second classification result of the object 270 calculated in the external electronic devices 103 and 105 based on the feature map. When executed by the processor 221, the instructions enable the electronic device 101 to determine whether the vehicle 210 is traveling as a leading vehicle based on the first and second classification results.

[0242] When executed by the processor 221, the instructions enable the electronic device 101 to identify whether the first classification result is correct based on the first classification result and the second classification result. When executed by the processor 221, the instructions enable the electronic device 101 to determine that the vehicle 210 is traveling as a rear vehicle based on the first classification result being incorrect. When executed by the processor 221, the instructions enable the electronic device 101 to determine that the vehicle 210 is traveling as a front vehicle based on the first classification result being correct.

[0243] When executed by the processor 221, the instructions enable the electronic device 101 to identify the accuracy rates of the electronic device 101 and the external electronic devices 103 and 105 based on the first classification result and the second classification result. When executed by the processor 221, the instructions enable the electronic device 101 to determine the vehicle (e.g., vehicle 230) in which the electronic device corresponding to the highest accuracy rate (e.g., electronic device 103) is mounted as the leading vehicle.

[0244] When executed by the processor 221, the instructions enable the electronic device 101 to identify a communication environment. When executed by the processor 221, the instructions enable the electronic device 101 to send the first classification result, in addition to the feature map, to the external electronic devices 103 and 105 via the communication circuit 211 based on a communication rate corresponding to the communication environment being lower than a reference communication rate. When executed by the processor 221, the instructions enable the electronic device 101 to send the feature map and the first classification result to the external electronic devices 103 and 105 via the communication circuit 211 based on a communication rate corresponding to the communication environment being higher than a reference communication rate.

[0245] When executed by the processor 221, the instructions enable the electronic device 101 to obtain lane line width information of the road on which the vehicle 210 is traveling via the communication circuit 211. When executed by the processor 221, the instructions enable the electronic device 101 to identify a first number of pixels representing the width of the lane line contained in the image and a second number of pixels in the width direction of the object 270. When executed by the processor 221, the instructions enable the electronic device 101 to identify the size of the object 270 in the width direction based on the ratio between the first number of pixels and the second number of pixels and the lane line width information.

[0246] When the processor 221 executes the instructions, the electronic device 101 is enabled to identify the type represented by the majority of the first classification result and the second classification result as the type of the object 270 .

[0247] The electronic device 105 described above may include: a camera 245; a communication circuit 215; a memory 235 storing a neural network model 400 and instructions; and a processor 225 operatively connected to the camera 245, the communication circuit 215, and the memory 235. The instructions, when executed by the processor 225, enable the electronic device 105 to acquire an image 600 via the camera 245. The instructions, when executed by the processor 225, enable the electronic device 105 to use the image 600 and the neural network model 400 to identify an area in the image 600 containing a leading vehicle 230. The instructions, when executed by the processor 225, enable the electronic device 105 to transmit, via the communication circuit 215, the remaining areas 730 and 735 of the image 600, excluding the area 720, to an external electronic device contained in a subsequent vehicle 250 in which the electronic device 105 is mounted.

[0248] When executed by the processor 225, the instructions enable the electronic device 105 to obtain a feature map from the external electronic device 101 mounted on the lead vehicle 210 via the communication circuit 215. When executed by the processor 225, the instructions enable the electronic device 105 to obtain a classification result of the object 270 associated with the feature map using the feature map and the neural network model 400. When executed by the processor 225, the instructions enable the electronic device 105 to transmit the classification result to the external electronic device 101 via the communication circuit 215.

[0249] When executed by the processor 225, the instructions enable the electronic device 105 to identify the object 270 in the image 600 using the image 600 and the neural network model 400. When executed by the processor 225, the instructions enable the electronic device 105 to identify the potential for collision between each of vehicles 210 and 230 other than the vehicle 250 in which the electronic device 105 is mounted and the object 270. When executed by the processor 225, the instructions enable the electronic device 105 to transmit, via the communication circuit 215, information related to the identified object 270 to the external electronic device 103 mounted on the first vehicle 230 of the other vehicles 210 and 230 with the highest potential for collision.

[0250] When the instruction is executed by the processor 225, the electronic device 105 can send relevant information of the identified object 270 to the external electronic device 101 carried by the remaining vehicles 210 through the communication circuit 215 after sending relevant information of the identified object 270 to the external electronic device 103 carried by the first vehicle 230.

[0251] The method described above can be performed by an electronic device including a camera 241 and a communication circuit 211, and may include the following operations. The method may acquire an image 300 through the camera 241. The method may use the image 300 and the neural network model 400 to identify a first classification result of the object 270 contained in the image 300. The method may transmit, through the communication circuit 211, a feature map of the neural network model 400 based on the image 300 to an external electronic device 103 or 105 contained in a subsequent vehicle 230 or 250 on which the electronic device 101 is loaded with the vehicle 210. The method may acquire, through the communication circuit 211, from the external electronic device 103 or 105 a second classification result of the object 270 calculated in the external electronic device 103 or 105 based on the feature map. The method may determine whether the vehicle 210 is to be driven as the preceding vehicle 210 based on the first and second classification results.

[0252] The method may determine whether the first classification result is correct based on the first classification result and the second classification result. The method may determine that the vehicle 210 is traveling as the rear vehicle 210 based on the first classification result being incorrect. The method may determine that the vehicle 210 is traveling as the front vehicle 210 based on the first classification result being correct.

[0253] The method can identify the accuracy rates of the electronic device 210 and the external electronic devices 230 and 250 based on the first classification result and the second classification result. The method can determine the vehicle (e.g., vehicle 230) carrying the electronic device (e.g., electronic device 103) corresponding to the highest accuracy rate as the leading vehicle.

[0254] The method can identify a communication environment. The method can send the first classification result, excluding the characteristic graph, to the external electronic device 103 or 105 via the communication circuit 211 based on a communication rate corresponding to the communication environment being lower than a reference communication rate. The method can send the characteristic graph and the first classification result to the external electronic device 103 or 105 via the communication circuit 211 based on a communication rate corresponding to the communication environment being higher than a reference communication rate.

[0255] The method may obtain lane width information of the road on which the vehicle 210 is traveling via the communication circuit 211. The method may identify a first number of pixels representing the width of the lane and a second number of pixels representing the width of the object 270 in the image 300. The method may identify the size of the object 270 in the width based on the ratio between the first number of pixels and the second number of pixels and the lane width information.

[0256] The method may identify the type represented by the majority of the first classification results and the second classification results as the type of the object 270 .

[0257] The method described above can be performed by an electronic device including a camera 245 and a communication circuit 215 and can include the following operations: The method can capture an image 600 using the camera 245. The method can use the image 600 and the neural network model 400 to identify an area in the image 600 that includes the preceding vehicle 230. The method can transmit, via the communication circuit 215, the remaining areas of the image 600, excluding the preceding area, to an external electronic device included in a subsequent vehicle 250 in which the electronic device 105 is mounted.

[0258] The method may obtain a feature map from an external electronic device mounted on a leading vehicle via the communication circuit 215 . The method may utilize the feature map and the neural network model 400 to obtain a classification result for the object 270 associated with the feature map. The method may transmit the classification result to the external electronic device via the communication circuit 215 .

[0259] The non-transitory computer readable storage medium (non-transitory computer readable storage medium) as described above can be used to store one or more programs. When the one or more programs are executed by the processor 221 in the electronic device 101 having the camera 241 and the communication circuit 211, the electronic device 101 can obtain the image 300 through the camera 241. When the one or more programs are executed by the processor 221, the electronic device 101 can use the image 300 and the neural network model 400 to identify the first classification result of the object 270 contained in the image 300. When the one or more programs are executed by the processor 221, the electronic device 101 can send the feature map of the neural network model 400 based on the image 300 to the external electronic device 103, 105 contained in the subsequent vehicle 230, 250 on which the electronic device 101 is carried by the vehicle 210 through the communication circuit 211. When executed by the processor 221, the one or more programs enable the electronic device 101 to obtain, from the external electronic devices 103 and 105 via the communication circuit 211, a second classification result of the object 270 calculated in the external electronic devices 103 and 105 based on the feature map. When executed by the processor 221, the one or more programs enable the electronic device 101 to determine whether the vehicle 210 is traveling as a leading vehicle based on the first and second classification results.

[0260] The non-transitory computer-readable storage medium described above can be used to store one or more programs. When executed by the processor 225 in the electronic device 105 having the camera 245 and the communication circuit 215, the one or more programs enable the electronic device 105 to obtain the image 600 through the camera 245. When executed by the processor 225, the one or more programs enable the electronic device 105 to use the image 600 and the neural network model 400 to identify the area in the image 600 that includes the leading vehicle 230. When executed by the processor 225, the one or more programs enable the electronic device 105 to transmit, via the communication circuit 215, the remaining areas 730 and 735 of the image 600, excluding the area 720, to an external electronic device included in a subsequent vehicle 250 in which the electronic device 105 is mounted.

[0261] The various embodiments in this document and the terms used therein should not limit the technical features in this document to specific embodiments, but should be understood to include various modifications, equivalents or substitutes of these embodiments. With respect to the description of the drawings, similar figure marks may be used for similar or related constituent elements. Unless otherwise clearly indicated by the relevant context, the singular form of the noun corresponding to the item may include one or more of the above-mentioned items. In this document, each of phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include items related to any of the items listed in these phrases, or all possible combinations thereof. Terms such as "first", "second" or "first", "second" are only used to distinguish a constituent element from other constituent elements, and do not limit these constituent elements in other aspects (such as importance or order). If a certain (for example, a first) component is “functionally” or “communicatively” coupled or connected with another (for example, a second) component, or in the absence of these terms, it means that any of the above components can be connected to the other component directly (for example, by wire), wirelessly, or through a third component.

[0262] In the specific embodiments of the present disclosure described above, the constituent elements included in the disclosure are expressed in singular or plural form according to the description of the specific embodiment. However, the singular or plural expression is selected according to the specific circumstances for the convenience of explanation, and the present disclosure is not limited to singular or plural constituent elements. Even constituent elements expressed in plural form can be composed of a single element, and constituent elements expressed in singular form can also be composed of multiple elements.

[0263] According to an embodiment, one or more constituent elements or operations in the above-mentioned constituent elements can be omitted, or one or more other constituent elements or operations can be added. Alternatively or additionally, multiple constituent elements (for example, modules or programs) can be integrated into one constituent element. In this case, the constituent element after integration can perform these functions in a manner that the corresponding constituent elements in the multiple constituent elements before integration perform the same or similar functions. According to an embodiment, the operations performed by modules, programs or other constituent elements can be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the above-mentioned operations can be performed in different orders, omitted, or one or more other operations can be added.

[0264] On the other hand, although specific embodiments have been described in the detailed description of the present disclosure, it is apparent that various modifications can be made without departing from the scope of the present disclosure.

Claims

1. An electronic device comprising: Camera; Communication circuits; Memory for storing neural network models and instructions; as well as a processor operatively connected to the camera, the communication circuit, and the memory, When the instructions are executed by the processor, the electronic device is enabled to: Acquire an image through the camera; Using the image and the neural network model, identifying a first classification result of an object contained in the image; transmitting, via the communication circuit, a feature map of the neural network model based on the image to an external electronic device included in a subsequent vehicle in which the electronic device is mounted; acquiring, from the external electronic device through the communication circuit, a second classification result of the object calculated in the external electronic device based on the feature map; and Based on the first classification result and the second classification result, it is determined whether the vehicle is traveling as a leading vehicle.

2. The electronic device according to claim 1, wherein When executed by the processor, the instructions enable the electronic device to: Based on the first classification result and the second classification result, identifying whether the first classification result is correct; Based on the first classification result being an error, determining that the vehicle should travel as a rear vehicle; Based on the first classification result being correct, it is determined that the vehicle travels as the leading vehicle.

3. The electronic device according to claim 1, wherein: When executed by the processor, the instructions enable the electronic device to: an accuracy rate of identifying the electronic device and the external electronic device based on the first classification result and the second classification result; The vehicle on which the electronic device corresponding to the highest accuracy rate among the accuracy rates is mounted is determined as the leading vehicle.

4. The electronic device according to claim 1, wherein: When executed by the processor, the instructions enable the electronic device to: Identify the communication environment; Based on the communication rate corresponding to the communication environment being lower than a reference communication rate, sending the first classification result except the feature map to the external electronic device through the communication circuit; Based on the communication rate corresponding to the communication environment exceeding a reference communication rate, the feature map and the first classification result are sent to the external electronic device through the communication circuit.

5. The electronic device according to claim 1, wherein: When executed by the processor, the instructions enable the electronic device to: Acquiring lane line width information of the road on which the vehicle is traveling through the communication circuit; Identifying a first number of pixels of the width of the lane line and a second number of pixels of the object in a width direction included in the image; Based on the ratio between the first number of pixels and the second number of pixels and the width information of the lane line, a size of the object in the width direction is identified.

6. The electronic device according to claim 1, wherein: When executed by the processor, the instructions enable the electronic device to: The type represented by the majority of the first classification results and the second classification results is identified as the type of the object.

7. An electronic device comprising: Camera; Communication circuits; Memory for storing neural network models and instructions; as well as a processor operatively connected to the camera, the communication circuit, and the memory, When the instructions are executed by the processor, the electronic device is enabled to: Acquire an image through the camera; Using the image and the neural network model, identifying an area in the image containing a front vehicle; The remaining areas of the image except the area are transmitted to an external electronic device included in a subsequent vehicle after the vehicle in which the electronic device is mounted, through the communication circuit.

8. The electronic device according to claim 7, wherein: When executed by the processor, the instructions enable the electronic device to: acquiring a characteristic map from an external electronic device mounted on a lead vehicle via the communication circuit; Obtaining a classification result of an object related to the feature map using the feature map and the neural network model; The classification result is sent to the external electronic device through the communication circuit.

9. The electronic device according to claim 7, wherein: When executed by the processor, the instructions enable the electronic device to: identifying an object in the image using the image and the neural network model; identifying the possibility of collision between each of vehicles other than the vehicle in which the electronic device is mounted and the object; The information related to the identified object is sent via the communication circuit to an external electronic device carried by a first vehicle with the highest collision possibility among the other vehicles.

10. The electronic device according to claim 9, wherein: When executed by the processor, the instructions enable the electronic device to: After sending the relevant information of the identified object to the external electronic device carried by the first vehicle through the communication circuit, the relevant information of the identified object is sent to the external electronic devices carried by the remaining vehicles.

11. A method performed by an electronic device comprising a camera and a communication circuit, comprising the following operations: Acquire an image through the camera; Using the image and the neural network model, identifying a first classification result of an object contained in the image; transmitting, via the communication circuit, a feature map of the neural network model based on the image to an external electronic device included in a subsequent vehicle in which the electronic device is mounted; acquiring, from the external electronic device through the communication circuit, a second classification result of the object calculated in the external electronic device based on the feature map; as well as Based on the first classification result and the second classification result, it is determined whether the vehicle is traveling as a leading vehicle.

12. The method according to claim 11, wherein The following operations are included: Based on the first classification result and the second classification result, identifying whether the first classification result is correct; Based on the first classification result being an error, determining that the vehicle should travel as a rear vehicle; as well as Based on the first classification result being correct, it is determined that the vehicle travels as the leading vehicle.

13. The method according to claim 11, wherein The following operations are included: Based on the first classification result and the second classification result, the accuracy rate of identifying the electronic device and the external electronic device; and The vehicle on which the electronic device corresponding to the highest accuracy rate among the accuracy rates is mounted is determined as the leading vehicle.

14. The method according to claim 11, wherein The following operations are included: Identify the communication environment; Based on the communication rate corresponding to the communication environment being lower than a reference communication rate, sending the first classification result except the feature map to the external electronic device through the communication circuit; as well as Based on the communication rate corresponding to the communication environment exceeding a reference communication rate, the feature map and the first classification result are sent to the external electronic device through the communication circuit.

15. The method according to claim 11, wherein The following operations are included: Acquiring lane line width information of the road on which the vehicle is traveling through the communication circuit; Identifying a first number of pixels of the width of the lane line and a second number of pixels of the object in a width direction included in the image; as well as Based on the ratio between the first number of pixels and the second number of pixels and the width information of the lane line, a size of the object in the width direction is identified.

16. The method according to claim 11, wherein The following operations are included: The type represented by the majority of the first classification results and the second classification results is identified as the type of the object.

17. A method performed by an electronic device comprising a camera and a communication circuit, comprising the following operations: Acquire an image through the camera; Using the image and the neural network model, identifying an area in the image containing a front vehicle; and The remaining areas of the image except the area are transmitted to an external electronic device included in a subsequent vehicle after the vehicle in which the electronic device is mounted, through the communication circuit.

18. The method according to claim 17, wherein The following operations are included: acquiring a characteristic map from an external electronic device mounted on a lead vehicle via the communication circuit; Obtaining a classification result of an object related to the feature map using the feature map and the neural network model; as well as The classification result is sent to the external electronic device through the communication circuit.

19. A non-transitory computer-readable storage medium for storing one or more programs that, when executed by a processor in an electronic device having a camera and a communication circuit, Acquire an image through the camera; Using the image and the neural network model, identifying a first classification result of an object contained in the image; transmitting, via the communication circuit, a feature map of the neural network model based on the image to an external electronic device included in a subsequent vehicle in which the electronic device is mounted; acquiring, from the external electronic device through the communication circuit, a second classification result of the object calculated in the external electronic device based on the feature map; Based on the first classification result and the second classification result, it is determined whether the vehicle is traveling as a leading vehicle.

20. A non-transitory computer-readable storage medium for storing one or more programs that, when executed by a processor in an electronic device having a camera and a communication circuit, Acquire an image through the camera; Using the image and the neural network model, identifying an area in the image containing a front vehicle; The remaining areas of the image except the area are transmitted to an external electronic device included in a subsequent vehicle after the vehicle in which the electronic device is mounted, through the communication circuit.

Images