Analog signal output method and electronic apparatus for performing the method

Abstract

To provide an electronic device and a method of outputting an analog signal using the electronic device installed in a vehicle.SOLUTION: Provided is an electronic device configured to receive an analog signal for an original image captured using a target camera mounted on a vehicle, convert the analog signal for the original image into a digital signal based on a resolution of the original image, generate a digital image based on the digital signal, determine a target object in the digital image using one or more image recognition models, generate a synthesized image by synthesizing a computer graphic for the target object with the digital image, convert the synthesized image into an analog signal, and transmit the analog signal for the synthesized image to an analog signal-receiving device installed in the vehicle.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method and system for outputting an analog signal of a synthetic video using an electronic device.

Background Art

[0002] A function is provided to output and store peripheral video captured using a camera provided in a vehicle for safe vehicle operation via a vehicle internal device. There is a need for a means to notify the driver not only by assisting in ensuring the driver's field of view while driving but also by detecting and determining a dangerous situation of the vehicle and displaying it on the captured video. There is a need to send a video notifying of a dangerous situation to a device in the vehicle by installing an electronic device in a data transmission path without replacing the existing shooting and monitoring means provided in the vehicle.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An electronic device according to an embodiment is to provide an analog signal output method using an electronic device provided in a vehicle.

Means for Solving the Problems

[0004] An analog signal output method of a synthetic video executed by an electronic device provided in a vehicle according to an embodiment includes an operation of receiving an analog signal for an original video captured using a target camera mounted on the vehicle, an operation of converting the analog signal for the original video into a digital signal based on the resolution of the original video, an operation of generating a digital video based on the digital signal, an operation of determining a target object in the digital video using one or more video recognition models, an operation of generating a synthetic video by synthesizing computer graphics for the target object into the digital video, an operation of converting the synthetic video into an analog signal, and an operation of transmitting the analog signal for the synthetic video to an analog signal receiving device provided in the vehicle.

[0005] The operation of generating a digital image based on the digital signal may include the operation of generating the digital image having the format of raw data or YUV data based on the digital signal.

[0006] The operation of determining a target object in the digital image using one or more image recognition models may include: generating first processing data for a deep learning image recognition method and second processing data for a computer vision image recognition method based on the digital image; generating deep learning recognition data for the first processing data using a deep learning image recognition model; generating computer vision recognition data for the second processing data using a computer vision image recognition model; and determining a target object in the digital image based on the deep learning recognition data and the computer vision recognition data.

[0007] The operation of generating the first processed data and the second processed data based on the digital image may include the operation of generating the first processed data and the second processed data by applying at least one of the following processes to the digital image: color format conversion, filtering, noise reduction, cropping, or scaling.

[0008] The operation of generating the deep learning recognition data for the first processed data using the deep learning image recognition model may include the operation of determining target object information for the first processed data using the deep learning image recognition model, and the operation of generating the deep learning recognition data by determining whether or not the target object in the digital image corresponding to the current frame has been detected based on the previous object information for previous frames within a predetermined frame range and the target object information.

[0009] The target object information may include at least one of the following: the type of the target object, its coordinates, its shape, or a score indicating the accuracy of recognition.

[0010] The operation of generating the deep learning recognition data by determining whether or not the target object in the digital image is detected may include the operation of determining the detection of the target object in the digital image if the score for the target object in the current frame is less than a predetermined first threshold, and the average value of the previous scores for the target object in the previous frames is greater than or equal to a predetermined second threshold.

[0011] The operation of generating the computer vision recognition data for the second processed data using the computer vision image recognition model is an operation of generating the computer vision recognition data for the second processed data using the computer vision image recognition model, which includes correction information based on camera parameters for the camera.

[0012] The operation of generating a composite image by combining computer graphics for the target object with the digital image may include the operation of generating a marker image layer as the computer graphics that displays the position of the target object based on the target object information included in the deep learning recognition data and the correction information included in the computer recognition data, and the operation of generating the composite image by combining the digital image and the computer graphics.

[0013] The analog signal receiving device may be a black box module.

[0014] The analog signal output method for the synthesized video may further include the operation of receiving vehicle operation information and the operation of determining whether or not there are any risk factors for the current state of the vehicle based on the vehicle operation information and the deep learning recognition data.

[0015] The operation of receiving the vehicle's operation information may include at least one of the following: receiving the vehicle's operation information from an operation information relay module, or receiving the vehicle's location information from a GPS module.

[0016] The analog signal output method for the composite video may further include an operation to output a danger warning if the danger element is present.

[0017] An electronic device for outputting an analog signal of a composite image according to one embodiment includes: a first conversion unit that receives an analog signal of an original image captured using a target camera mounted on a vehicle; a signal processing unit that converts the analog signal of the original image into a digital signal based on the resolution of the original image; an image processing unit that generates a digital image based on the digital signal; an image recognition unit that determines a target object in the digital image using one or more image recognition models; and a hazard detection software unit that generates a composite image by compositing computer graphics of the target object into the digital image, converts the composite image into an analog signal, and transmits the analog signal of the composite image to an analog signal receiving device provided on the vehicle.

[0018] The image recognition unit can perform the following operations: generate first processing data for a deep learning image recognition method and second processing data for a computer vision image recognition method based on the digital image; generate deep learning recognition data for the first processing data using a deep learning image recognition model; generate computer vision recognition data for the second processing data using a computer vision image recognition model; and determine a target object in the digital image based on the deep learning recognition data and the computer vision recognition data.

[0019] The operation of generating the deep learning recognition data for the first processed data using the deep learning image recognition model may include the operation of determining target object information for the first processed data using the deep learning image recognition model, and the operation of generating the deep learning recognition data by determining whether or not the target object in the digital image corresponding to the current frame has been detected based on the previous object information for previous frames within a predetermined frame range and the target object information.

[0020] The target object information may include at least one of the following: the type of the target object, its coordinates, its shape, or a score indicating the accuracy of recognition.

[0021] The operation of generating the deep learning recognition data by determining whether or not the target object in the digital image is detected may include the operation of determining the detection of the target object in the digital image if the score for the target object in the current frame is less than a predetermined first threshold, and the average value of the previous scores for the target object in the previous frames is greater than or equal to a predetermined second threshold.

[0022] The operation of generating a composite video by synthesizing computer graphics for the target object into the digital video may include an operation of generating a marker video layer that displays the position of the target object as the computer graphics based on the target object information included in the deep learning recognition data and the correction information included in the computer recognition data, and an operation of generating the composite video by synthesizing the digital video and the computer graphics.

Advantages of the Invention

[0023] According to the present invention, an electronic device can provide an analog signal output method using an electronic device provided in a vehicle.

Brief Description of the Drawings

[0024] [Figure 1] It is a diagram showing an analog signal output system according to an embodiment. [Figure 2] It is a block diagram of an electronic device according to an embodiment. [Figure 3] It is a flowchart of an analog signal output method of a composite video according to an embodiment. [Figure 4] It is a flowchart of a target object determination method according to an embodiment. [Figure 5] It is a flowchart of a deep learning recognition data generation method according to an embodiment. [Figure 6] It is a flowchart of a composite video generation method according to an embodiment. [Figure 7] It is a diagram for explaining a composite video according to an example. [Figure 8] It is a flowchart for explaining reception of operation information according to an embodiment. [Figure 9] It is a flowchart of a danger warning output method according to an embodiment. [Figure 10] It is a block diagram of an electronic device according to an embodiment. [Modes for carrying out the invention]

[0025] The specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and can be modified in various ways. Therefore, the embodiments are not limited to the specific disclosure, and the scope of this specification includes modifications, equivalents, or substitutions that are part of the technical idea.

[0026] Terms such as "first" or "second" may be used to describe multiple components, but such terms should be interpreted solely for the purpose of distinguishing one component from others. For example, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0027] When it is mentioned that one component is “linked” or “connected” to another component, it should be understood that it is directly linked to or connected to the other component, but that other components may be present in between.

[0028] A singular expression includes plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “has” indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood not to presuppose the existence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0029] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which this embodiment belongs. Commonly used, predefined terms should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as ideal or overly formal unless expressly defined herein.

[0030] The embodiments will be described in detail below with reference to the attached drawings. When describing with reference to the drawings, the same components will be given the same reference numerals regardless of the reference numerals used in the drawings, and redundant explanations will be omitted.

[0031] Figure 1 shows an analog signal output system according to one embodiment.

[0032] An analog signal output system 100 according to one embodiment includes an electronic device 110, a camera 120, an analog signal receiving device 130, and an external device 140. The electronic device 110, camera 120, analog signal receiving device 130, and external device 140 may be installed in a vehicle (not shown).

[0033] According to one embodiment, camera 120 refers to a target camera connected to the electronic device 110, which is one or more camera modules located inside and / or outside the vehicle. The analog signal output system 100 includes an electronic device module corresponding to each camera module located in the vehicle. For example, camera 120 is a camera that photographs the front or rear of the vehicle. For example, camera 120 may be a camera that photographs the left side or the right side of the vehicle.

[0034] According to one embodiment, the camera 120 outputs an analog signal for the captured original video, and the analog signal receiving device 130 can output the received analog signal in real time via a display or store it in memory. For example, the analog signal receiving device 130 may represent a black box module installed in a vehicle. The electronic device 110 may be installed between the camera 120 and the analog signal receiving device 130. The electronic device 110 receives the analog signal output by the camera 120, generates a digitally composited video based on the received analog signal, and transmits an analog signal for the digitally composited video to the analog signal receiving device 130. For example, the camera 120 outputs the analog signal for the captured original video in CVBS (Composite Video Baseband Signal), TVI (Turbo Video Interface), or AHD (Analog High Definition) format, and the resolution of the original video may be FHD (1920x1080), HD (1280x720), or CVBS (720x480i). The electronic device 110 can transmit an analog signal to the analog signal receiving device 130 for a digitally composited image having the same format and resolution as the analog signal output by the camera 120.

[0035] According to one embodiment, the electronic device 110 includes a first conversion unit 150, a signal processing unit 160, a video processing unit 170, a video recognition unit 180, and a hazard detection software unit 190. The electronic device 110 will be described in detail below with reference to Figure 2.

[0036] According to one embodiment, the external device 140 includes an operation information relay module 142 and a GPS module 144. The analog signal output system 100 may or may not selectively include the operation information relay module 142 and / or the GPS module 144. The electronic device 110 receives vehicle operation information from the external device 140. The operation information includes vehicle operation information and location information. The vehicle operation information may include one or more of the following: vehicle left / right turn signal lamp information, reverse gear information, brake information, or speed information.

[0037] According to one embodiment, the electronic device 110 can receive vehicle operation information from the operation information relay module 142. The electronic device 110 may also receive vehicle location information from the GPS module 142 (directly or via the operation information relay module 142).

[0038] According to one embodiment, the operation information relay module 142 may acquire information regarding whether the turn signal lamps are on, whether the vehicle is in reverse gear, or the operating state of the brake pedal, based on analog signals that can be obtained via the power cables of the vehicle's turn signal lamps, reverse lamps, or brake lamps. If the operation information relay module 142 is not connected to the GPS module 144, the operation information relay module 142 can acquire speed information via a power cable connected to a speed sensor that measures the vehicle's speed by counting the vehicle's rotations. If the operation information relay module 142 is connected to the GPS module 144, the operation information relay module 142 can acquire speed information by calculating the vehicle's travel distance and time interval using the position information received from the GPS module 144. For example, the operation information relay module 142 can combine information obtained via the power cables of additional sensors such as speed sensors or other pressure sensors with information obtained via the power cables of the brake lamps to determine the force applied to the brake pedal and the duration of brake application.

[0039] According to one embodiment, the operation information relay module 142 can receive various information related to vehicle hazard detection from the vehicle's OBD (On-Board Diagnostics) system via CAN (Controller Area Network) communication. The operation information relay module 142 can receive not only operational information including one or more of the vehicle's left / right turn signal lamp information, reverse gear information, brake information, or speed information, but also the vehicle's mileage, fuel consumption, driving time, vehicle position, engine status, etc., and is not limited to this description. As a result, the electronic device can receive vehicle operation information from the OBD system or power cable via the operation information relay module 142.

[0040] Figure 2 is a block diagram of an electronic device according to one embodiment.

[0041] According to one embodiment, the electronic device 110 includes a first conversion unit 150, a signal processing unit 160, a video processing unit 170, a video recognition unit 180, and a hazard detection software unit 190.

[0042] The first conversion unit 150 receives analog signals for original video footage captured from one or more camera modules installed in the vehicle. For example, the first conversion unit 150 may receive analog signals in a format and resolution predetermined by the user. For example, the first conversion unit 150 may estimate the resolution of the received analog signal. Based on the format or resolution of the analog signal, the first conversion unit 150 converts the received analog signal into a digital signal. For example, the first conversion unit 150 may convert the analog signal into a digital signal conforming to the BT.656 (ITU-BT.656 or CCIR-656) standard and transmit the converted digital signal to the signal processing unit 160.

[0043] The signal processing unit 160 is an Image Signal Processor (ISP). The signal processing unit 160 can generate digital video in the form of raw data or YUV data based on digital signals. The signal processing unit 160 may buffer the generated digital video. The digital video can be transmitted to the video processing unit 170 and the compositing unit 192.

[0044] The video processing unit 170 can generate first processing data for a deep learning video recognition system and second processing data for a computer vision video recognition system based on digital video. The video processing unit 170 may generate the first and second processing data by applying at least one of the following processes to the digital video: color format conversion (RGB or YUV), filtering, noise reduction, cropping, or scaling.

[0045] The image recognition unit 180 includes a deep learning recognition unit 182 and a computer vision recognition unit 184. The deep learning recognition unit 182 includes a deep learning unit 182-1 and a data processing unit 182-2. The computer vision recognition unit 184 includes a recognition assistance unit 184-1 and a correction unit 184-2. The recognition assistance unit 184-1 may be included selectively.

[0046] According to one embodiment, the video recognition unit 180 can generate deep learning recognition data and computer vision recognition data for DSM (Driver Status Monitoring) and BSD (Blind Spot Detection) from the captured video.

[0047] According to one embodiment, the deep learning recognition unit 182 generates deep learning recognition data for the first processing data using a pre-stored and / or trained deep learning image recognition model. The deep learning unit 182-1 determines target object information for the first processing data using the deep learning image recognition model. The target object information may include at least one of the following: the type of target object, coordinates, shape, or a score indicating the accuracy of recognition. The target object represents an object that causes a dangerous situation for a vehicle among the various objects recognized by the deep learning image recognition model. The data processing unit 182-2 can generate deep learning recognition data by determining whether or not a target object has been detected in the digital image.

[0048] According to one embodiment, the deep learning recognition unit 182 can use a deep learning recognition model that has been specifically trained according to the installation position of the target camera to which the electronic device 110 is connected. For example, if the electronic device 110 is connected to a target camera that photographs the front of the vehicle, the electronic device 110 may include a deep learning recognition unit 182 that uses a deep learning recognition model specialized for recognizing the front-facing image. For example, if the electronic device 110 is connected to a target camera that photographs the rear of the vehicle, the electronic device 110 may include a deep learning recognition unit 182 that uses a deep learning recognition model specialized for recognizing the rear-facing image.

[0049] According to one embodiment, the computer vision recognition unit 184 can generate computer vision recognition data for second processing data using a pre-stored and / or trained computer vision image recognition model. The recognition assistance unit 184-1 may re-determine target object information for the second processing data using the computer vision image recognition model to complement the deep learning recognition data generated by the deep learning recognition unit 182. For example, the recognition assistance unit 184-1 may re-determine target object information to detect target objects that the deep learning recognition unit 182 cannot detect. The correction unit 184-2 may generate computer vision recognition data for the second processing data (or for second processing data in which target object information has been re-determined) that includes correction information based on camera parameters of a camera (e.g., camera 120 in Figure 1). The camera parameters may include the characteristics of the sensors and lenses constituting the camera or the coordinates and installation angle of the camera mounted on the vehicle.

[0050] The hazard detection software unit 190 collects and stores deep learning recognition data, computer vision recognition data, camera parameters, and various information related to vehicle hazard detection (e.g., measured distance between the vehicle and target object, vehicle acceleration, vehicle excelrate motion, vehicle operation information, etc.), enabling comprehensive hazard detection based on all the data. The hazard detection software unit 190 includes a synthesis unit 192, a hazard detection unit 194, and a second conversion unit 196.

[0051] The synthesis unit 192 can generate a marker image layer as a computer graphic that displays the position of a target object based on deep learning recognition data and computer vision recognition data. The synthesis unit 192 can generate a composite image by combining the digital image generated by the signal processing unit 160 and the generated computer graphic. The generation of the composite image will be explained in detail with reference to Figure 7.

[0052] The hazard detection unit 194 determines whether or not there are hazardous elements in relation to the vehicle's current state based on vehicle operation information and deep learning recognition data. In determining the presence or absence of hazardous elements, computer vision recognition data including re-determined target object information and / or correction information may also be used.

[0053] The second conversion unit 196 can convert the composite video generated by the synthesis unit 192 into an analog signal. For example, the second conversion unit 196 can convert the composite video into an analog signal in CVBS or TVI format, and the resolution of the video may be FHD, HD, or CVBS. The second conversion unit 196 can convert the composite video into an analog signal having the same format and resolution as the analog signal output by the camera 120.

[0054] Figure 3 is a flowchart of an analog signal output method for a composite image according to one embodiment.

[0055] According to one embodiment, the following operations 310 to 390 are performed by an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0056] In operation 310, the electronic device receives an analog signal for the original video footage captured using a target camera mounted on the vehicle.

[0057] According to one embodiment, the electronic device can receive an analog signal in a format and resolution predetermined by the user. For example, the electronic device can receive an analog signal in CVBS, TVI, or AHD format for the captured original video, and the resolution of the video may be FHD, HD, or CVBS.

[0058] According to one embodiment, the electronic device can estimate the resolution of the received analog signal.

[0059] In operation 320, the electronic device converts the analog signal to the original video into a digital signal based on the resolution of the original video. The electronic device may convert the analog signal to the original video into a digital signal based on a predetermined or estimated resolution. For example, the electronic device may convert the analog signal into a digital signal conforming to the BT.656 (ITU-BT.656 or CCIR-656) standard.

[0060] In operation 330, the electronic device generates a digital image based on a digital signal. The electronic device may generate a digital image in the form of raw data or YUV data based on the digital signal. The electronic device may buffer the generated digital image.

[0061] In operation 340, the electronic device determines a target object in the digital image using one or more image recognition models. The method for determining the target object will be described in detail with reference to Figure 4.

[0062] In operation 350, the electronic device generates a composite image by compositing computer graphics for a target object with digital video. For example, the electronic device may generate a marker video layer as computer graphics that displays the position of the target object. The electronic device can generate a composite image by compositing digital video and computer graphics. For example, the electronic device may generate a composite image in which frame markers are displayed at the position of the target object.

[0063] In operation 360, the electronic device converts the composite image into an analog signal. The electronic device may also convert the composite image into an analog signal having the same format and resolution as the analog signal received in operation 310.

[0064] In operation 370, the electronic device transmits an analog signal for the composite image to an analog signal receiving device (for example, the analog signal receiving device 130 in Figure 1) installed in the vehicle. The analog signal receiving device receives the analog signal for the composite image and can output the image corresponding to the received analog signal in real time via a display or store it in memory.

[0065] In operation 380, the electronic device receives vehicle operation information. This vehicle operation information includes vehicle operation information and location information. The reception of vehicle operation information will be described in detail below with reference to Figure 8.

[0066] According to one embodiment, operation 390 is performed after operation 340 has been performed.

[0067] In operation 390, the electronic device determines whether there are any hazards to the vehicle's current state based on the vehicle's operational information and deep learning recognition data. For example, the electronic device determines that there are hazards to the vehicle's forward or reverse state if there is an object traveling straight ahead in the vehicle's direction of travel or an object located in the travel path. For example, the electronic device may determine that there are hazards to the vehicle's left-turn or right-turn state if an object is located in the vehicle's travel path in the left-turn or right-turn direction. For example, the electronic device may determine that there are hazards if there is an object within a certain distance that poses a collision risk based on the vehicle's speed and / or acceleration. For example, the electronic device can determine the presence or absence of hazards based on the real-time coordinates and type of the target object and the vehicle's driving operation.

[0068] According to one embodiment, operation 390 is performed after operation 420 shown in Figure 4 below.

[0069] Figure 4 is a flowchart of a method for determining a target object according to one embodiment.

[0070] According to one embodiment, the following operations 410 to 440 are performed after the operation 330 described above is performed with reference to Figure 3. Operations 410 to 440 can be performed by an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0071] In operation 410, the electronic device generates first processing data for a deep learning image recognition system and second processing data for a computer vision image recognition system based on the digital image.

[0072] According to one embodiment, the electronic device can generate first processed data and second processed data, respectively, by applying at least one of the following processes to a digital image: color format conversion, filtering, noise reduction, cropping, or scaling. The first processed data is generated in a format or type suitable for processing by the deep learning recognition unit of the electronic device (e.g., the deep learning recognition unit 182 in Figures 1 and 2), and the second processed data is generated in a format suitable for processing by the computer vision recognition unit of the electronic device (e.g., the computer vision recognition unit 184 in Figures 1 and 2).

[0073] In operation 420, the electronic device generates deep learning recognition data for the first processing data using a deep learning image recognition model. The deep learning recognition data includes whether or not a target object is detected in the digital image. The method for generating deep learning recognition data will be described in detail with reference to Figure 5.

[0074] In operation 430, the electronic device generates computer vision recognition data for the second processing data using a computer vision image recognition model. The electronic device generates computer vision recognition data for the second processing data using a computer vision image recognition model, which includes correction information based on camera parameters for the camera (e.g., camera 120 in Figure 1). The camera parameters may include the characteristics of the sensors and lenses constituting the camera, or the coordinates and installation angle of the camera mounted on the vehicle.

[0075] In operation 440, the electronic device determines a target object in the digital image based on deep learning recognition data and computer vision recognition data. The electronic device may determine at least one of the following: the type, coordinates, shape, or a score indicating the accuracy of recognition of the target object. The electronic device may determine a corrected target object based on camera parameters.

[0076] Figure 5 is a flowchart of a deep learning recognition data generation method according to one embodiment.

[0077] According to one embodiment, the following operations 510 and 520 are performed after the operation 410 described above is performed with reference to Figure 4. Operations 510 and 520 can be performed by an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0078] According to one embodiment, operation 510 is performed by the deep learning unit (for example, the deep learning unit 182-1 in Figure 2) of the deep learning recognition unit of the electronic device (for example, the deep learning recognition unit 182 in Figures 1 and 2), and operation 520 is performed by the data processing unit (for example, the data processing unit 182-2 in Figure 2) of the deep learning recognition unit.

[0079] In operation 510, the deep learning unit determines target object information for the first processing data using a deep learning image recognition model. The target object information may include at least one of the following: the type of target object, coordinates, shape, or a score indicating recognition accuracy. For example, the type of target object may include people, vehicles, bicycles, and other road / pedestrian walkway fixtures. For example, the shape of the target object may include the width and height of the target object.

[0080] According to one embodiment, the target object is an object that causes a dangerous situation for the vehicle among the various objects recognized by the deep learning image recognition model. For example, the deep learning unit may determine an object that is moving straight in the direction of travel of the vehicle or an object located along the travel path as the target object. Depending on the embodiment, the series of processes of detecting an object and determining the target object may be handled in one process or processed in one step.

[0081] According to one embodiment, if the score of the target object is less than a predetermined first threshold, the deep learning unit does not need to transmit the target object information to the data processing unit. Here, the data processing unit may recognize the score of the target object as 0. If the score of the target object is equal to or greater than the predetermined first threshold, the deep learning unit can transmit the target object information to the data processing unit.

[0082] In operation 520, the data processing unit can generate deep learning recognition data by determining whether or not a target object has been detected in the digital image corresponding to the current frame, based on previous object information and target object information for previous frames within a predetermined frame range.

[0083] According to one embodiment, the data processing unit can determine whether to detect a target object in the digital image if the score for the target object in the current frame is less than a predetermined first threshold, and the average value of the previous scores for the target object in previous frames is greater than or equal to a predetermined second threshold. The first and second thresholds may be the same or different. The average value of the scores in previous frames may represent an arithmetic mean or a weighted average, etc. For example, the closer the previous frame is to the current frame, the higher the weighted average value applied to the calculated average.

[0084] For illustrative purposes, let us assume that a given first and second threshold is 60%. For example, if the score for the target object in the current frame (e.g., a person or a bicycle) is 80%, the data processing unit may decide to detect the target object in the digital image. For example, if the score for the target object in the current frame is 50% and the average of previous scores for the same target object in previous frames is 70%, the data processing unit may decide to detect the target object in the digital image.

[0085] According to one embodiment, the data processing unit can determine the coordinates of the target object in the current frame based on the coordinates of the target object in previous frames within a predetermined frame range (or the average value of the coordinates in previous frames).

[0086] Figure 6 is a flowchart of a composite image generation method according to one embodiment.

[0087] According to one embodiment, the following operations 610 and 620 are performed after the operation 340 described above is performed with reference to Figure 3. Operations 610 and 620 can be performed by a composite unit (for example, the composite unit 192 in Figure 2) of an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0088] In operation 610, the synthesis unit generates a marker image layer as a computer graphic that displays the position of the target object based on the target object information included in the deep learning recognition data and the correction information included in the computer recognition data. The synthesis unit can generate a marker image layer as a computer graphic that displays the position of the target object using the coordinates of the target object and the correction information.

[0089] In operation 620, the compositing unit generates a composite image by combining digital video and computer graphics. For example, the compositing unit can combine digital video and computer graphics by overlapping them.

[0090] Figure 7 is a diagram illustrating a composite image as an example.

[0091] Figure 7 shows a screen where, when an electronic device (e.g., the electronic device 110 in Figures 1 and 2) is connected to a target camera that captures the area behind the vehicle, a composite image 700 is output via the display of an analog signal receiver (e.g., the analog signal receiver 130 in Figure 1) while the vehicle is reversing. For example, some of the recognized objects (e.g., trees, other vehicles, people, guardrails, etc.) (e.g., trees) may not be determined as target objects, while a person running in the direction of the vehicle's travel may be determined as a target object 710. The composite unit of the electronic device (e.g., the composite unit 192 in Figures 1 and 2) can generate a composite image 700 by compositing computer graphics 720 for the target object 710 onto the digital image captured from behind the vehicle.

[0092] The compositing unit may generate a marker image layer as a computer graphic 720 that displays the position of the target object 710, based on the target object information included in the deep learning recognition data and the correction information included in the computer vision recognition data. The compositing unit may also generate a marker image layer as a computer graphic 720 based on the re-determined target object information included in the computer vision recognition data. The compositing unit may generate a composite image 700 by compositing the digital image and the computer graphic 720. For example, the compositing unit may generate a composite image 700 in which a frame (or box) marker is displayed at the position of the target object 710. The composite image 700 may be generated such that the computer graphic 720 moves along with the movement of the target object 710 to display the position of the target object 710.

[0093] The information included in the composite image 700 is not limited to the examples shown in Figure 7, and the composite unit can generate the composite image 700 so that it shows the signal input status from any external device of the vehicle, the vehicle's position information, and so on.

[0094] Figure 8 is a flowchart illustrating the reception of operational information according to one embodiment.

[0095] According to one embodiment, operation 380 in Figure 3 includes the following operations 810 and 820. Operations 810 and 820 may be performed in whole or in part by an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0096] According to one embodiment, the vehicle operation information includes vehicle operation information and location information.

[0097] In operation 810, the electronic device receives vehicle operation information from the operation information relay module (for example, the operation information relay module 142 in Figure 1). The vehicle operation information may include one or more of the following: information on the left / right turn signals of the vehicle, reverse gear information, brake information, or speed information. The operation information relay module can obtain information on whether the turn signals are on, whether the vehicle is in reverse gear, or the operation status of the brake pedal, based on analog signals obtained via the power cables of the vehicle's turn signals, reverse lights, or brake lights. If the operation information relay module is not connected to the GPS module, the operation information relay module can obtain speed information via the power cable connected to a speed sensor that measures vehicle speed by counting the vehicle's rotations. If the operation information relay module is connected to the GPS module, the operation information relay module can obtain speed information by calculating the vehicle's travel distance and time interval using the position information received from the GPS module.

[0098] In operation 820, the electronic device receives vehicle location information from a GPS module (for example, GPS module 144 in Figure 1). The electronic device may receive vehicle location information from the GPS module directly or via a traffic information relay module.

[0099] According to one embodiment, the electronic device can calculate the speed by using location information to calculate the distance traveled and the time interval of the vehicle. However, if it is difficult for the GPS module to receive signals properly, such as when passing through a tunnel, the operation information relay module may acquire speed information via the power cable of the vehicle's speed sensor.

[0100] Figure 9 is a flowchart of a danger warning output method according to one embodiment.

[0101] According to one embodiment, the following operation 910 is performed after operation 390 in Figure 3. Operation 910 is performed by an electronic device (for example, the electronic device 110 in Figures 1 and 2).

[0102] In operation 910, the electronic device outputs a hazard warning if a hazardous element is present. For example, the electronic device may generate a hazard warning sound if a hazardous element is present. For example, the electronic device may output a pre-stored sound or warning via a speaker or buzzer.

[0103] Figure 10 is a block diagram of an electronic device according to one embodiment.

[0104] An electronic device 1000 according to one embodiment is shown in Figures 1 and 2, which is the electronic device 110. The electronic device 1000 includes a communication unit 1010, a memory 1020, a processor 1030, and an audio output unit 1040.

[0105] The communication unit 1010 is connected to a vehicle, a camera module installed in the vehicle, an external device, and an analog signal receiving device to transmit and receive data. The communication unit 1010 may also be connected to other external devices to transmit and receive data. In the following, the expression "transmitting and receiving A" means transmitting and receiving "information or data indicating A".

[0106] The communication unit 1010 may be implemented in the circuit network within the electronic device 1000. For example, the communication unit 1010 may include an internal bus and an external bus. For example, the communication unit 1010 may be an element connecting the electronic device 1000 and an external device. The communication unit 1010 may also be an interface. The communication unit 1010 receives data from an external device (e.g., a vehicle, an operation information relay module) and transmits the data to the processor 1030 and the memory 1020.

[0107] Memory 1020, as hardware for storing various types of data processed within the electronic device 1000, can store data processed by the processor 1030 and data to be processed. Memory 1020 may include at least one type of storage medium from among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory), RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk.

[0108] The processor 1030 controls the overall operation of the electronic device 1000. The processor 1030 may execute a set of instructions (e.g., software) stored in the memory 1020 that operates the electronic device 1000. The processor 1030 may be embodied in an array of logic gates, or in a combination of a general-purpose microprocessor and a memory containing the program implemented in that microprocessor. Furthermore, it will be understood by those ordinary skill in the art to which this embodiment belongs that it may be implemented in further forms of hardware.

[0109] The acoustic output unit 1040 provides the user with auditory information regarding any potential hazards to vehicle operation. For example, the acoustic output unit 1040 can convert electrical signals into acoustic signals and output them externally.

[0110] The embodiments described above are embodied in hardware components, software components, or combinations of hardware and software components. For example, the devices and components described in these embodiments are embodied using one or more general-purpose or special-purpose computers, such as a processor, controller, ALU (arithmetic logic unit), digital signal processor, microcomputer, FPA (field programmable array), PLU (programmable logic unit), microprocessor, or different devices that execute and respond to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the OS. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For convenience of understanding, the processing device may sometimes be described as being used as a single unit, but a person with ordinary skill in the art will understand that the processing device includes multiple processing elements and / or multiple types of processing elements. For example, the processing device includes multiple processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

[0111] Software includes computer programs, code, instructions, or a combination of one or more of these, which can configure a processing unit to operate as desired, or instruct the processing unit independently or in combination. Software and / or data can be permanently or temporarily embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, for interpretation by a processing unit or for providing instructions or data to a processing unit. Software can be distributed across a network of computer systems and stored and executed in a distributed manner. Software and data can be stored on a recording medium readable by one or more computers.

[0112] The method according to this embodiment is embodied in the form of program instructions that are implemented via various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, etc., individually or in combination. The recording medium and program instructions may be specifically designed and configured for the purposes of the present invention, or they may be known and usable by those skilled in the art who have technology in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like.

[0113] The hardware device described above may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0114] As described above, although embodiments have been illustrated with limited drawings, a person with ordinary skill in the art can apply various technical modifications and variations based on the above description. For example, the described technique may be performed in a different order than described, and / or the described system, structure, apparatus, circuit, and other components may be combined or assembled in a different manner than described, or replaced or substituted by other components or equivalents, and still achieve the desired results.

[0115] Therefore, other embodiments, other embodiments, and claims equivalent to those described below also fall within the scope of the claims.

Claims

1. A method for outputting an analog signal of a composite image, which is performed by an electronic device installed in a vehicle, The operation involves receiving analog signals for the original video footage captured using a target camera mounted on the vehicle, and An operation to convert the analog signal to the original video into a digital signal based on the resolution of the original video, The operation of generating a digital image based on the aforementioned digital signal, An operation to determine a target object in the digital image using one or more image recognition models, The operation of generating a composite image by compositing computer graphics of the target object onto the digital image, The operation of converting the aforementioned composite image into an analog signal, The operation of transmitting the analog signal for the composite image to an analog signal receiving device provided in the vehicle, Includes, The operation of determining a target object in the digital image using one or more image recognition models is: An operation to generate processed data by applying at least one of the following processes to the digital image: color format conversion, filtering, noise reduction, cropping, or scaling; An operation to generate first recognition data for the processing data using a learning model constructed by deep learning, An operation to generate second recognition data which includes correction information based on camera parameters for the camera for the processing data, An operation to determine a target object in the digital image based on the first recognition data and the second recognition data, A method for outputting analog signals of composite video, including the analog signal output method.

2. The analog signal output method for a composite image according to claim 1, wherein the operation of generating a digital image based on the digital signal includes the operation of generating the digital image having the format of RAW data or YUV data based on the digital signal.

3. The operation of generating the first recognition data for the processed data using the learning model constructed by deep learning is as follows: The operation of determining target object information for the processing data using the learning model constructed by the deep learning method, An operation to generate the first recognition data by determining whether or not the target object in the digital image corresponding to the current frame is detected based on previous object information for previous frames within a predetermined frame range and the target object information, The method for outputting an analog signal of a composite image according to claim 1, including the method described in claim 1.

4. The method for outputting an analog signal of a composite image according to claim 3, wherein the target object information may include at least one of the type, coordinates, shape, or score indicating the recognition accuracy of the target object.

5. The analog signal output method for a composite image according to claim 4, wherein the operation of generating the first recognition data by determining whether or not the target object in the digital image is detected includes the operation of determining the detection of the target object in the digital image if the score for the target object in the current frame is less than a predetermined first threshold and the average value of the previous scores for the target object in the previous frames is greater than or equal to a predetermined second threshold.

6. The operation of generating a composite image by compositing computer graphics for the target object onto the digital image includes the operation of generating a marker image layer as the computer graphics that displays the position of the target object based on the target object information included in the first recognition data and the correction information included in the second recognition data, The operation of generating the composite image by combining the digital image and the computer graphics, The method for outputting an analog signal of a composite image according to claim 1, including the method described in claim 1.

7. The analog signal output method for synthesized video according to claim 1, wherein the analog signal receiving device is a black box module.

8. The operation of receiving the vehicle's operation information, An operation to determine whether or not there are any risk factors for the current state of the vehicle based on the vehicle's operation information and the first recognition data, The method for outputting an analog signal of a composite image according to claim 1, further comprising:

9. The analog signal output method for a composite video according to claim 8, wherein the operation of receiving the vehicle operation information may include at least one of the following: the operation of receiving the vehicle operation information from the operation information relay module, or the operation of receiving the vehicle location information from the GPS module.

10. The method for outputting an analog signal of a composite image according to claim 8, further comprising the operation of outputting a danger warning if the aforementioned danger elements are present.

11. A computer-readable recording medium storing a program for performing the method described in any one of claims 1 to 10.

12. An electronic device for outputting analog signals of composite images, A first conversion unit that receives an analog signal for the original video captured using a target camera mounted on the vehicle, A signal processing unit that converts the analog signal to the original video into a digital signal based on the resolution of the original video, A video processing unit that generates a digital image based on the aforementioned digital signal, A video recognition unit that determines a target object in the digital image using one or more video recognition models, A hazard detection software unit generates a composite image by combining computer graphics of the target object with the digital image, converts the composite image into an analog signal, and transmits the analog signal of the composite image to an analog signal receiving device provided in the vehicle. Includes, The aforementioned video recognition unit, An operation to generate processed data by applying at least one of the following processes to the digital image: color format conversion, filtering, noise reduction, cropping, or scaling; An operation to generate first recognition data for the processing data using a learning model constructed by deep learning, An operation to generate second recognition data for the aforementioned processed data, An operation to determine a target object in the digital image based on the first recognition data and the second recognition data, Perform An operation information relay unit that receives operation information of the aforementioned vehicle, A hazard detection unit that determines whether or not there are hazardous elements to the current state of the vehicle based on the vehicle's operation information and the first recognition data, A danger alarm output unit outputs a danger alarm when the aforementioned danger elements are present, Electronic devices, including further electronic devices.

13. The operation of generating the first recognition data for the processed data using the learning model constructed by deep learning is as follows: The operation of determining target object information for the processing data using the learning model constructed by the deep learning method, An operation to generate the first recognition data by determining whether or not the target object in the digital image corresponding to the current frame is detected based on previous object information for previous frames within a predetermined frame range and the target object information, The electronic device according to claim 12, including the electronic device according to claim 12.

14. The electronic device according to claim 13, wherein the target object information may include at least one of the type, coordinates, shape, or score indicating the accuracy of recognition of the target object.

15. The electronic device according to claim 14, wherein the operation of generating the first recognition data by determining whether or not the target object in the digital image is detected includes the operation of determining the detection of the target object in the digital image if the score for the target object in the current frame is less than a predetermined first threshold and the average value of the previous scores for the target object in the previous frames is greater than or equal to a predetermined second threshold.

16. The operation of generating a composite image by compositing computer graphics of the target object onto the digital image is: Based on the target object information included in the first recognition data and the correction information included in the second recognition data, the operation of generating a marker video layer as a computer graphic that displays the position of the target object, The operation of generating the composite image by combining the digital image and the computer graphics, The electronic device according to claim 12, including the electronic device according to claim 12.

Images