Method, apparatus, and computer program for transmitting and receiving video data on basis of communication environment

Abstract

The present invention relates to a method, an apparatus, and a computer program for transmitting and receiving video data on the basis of a communication environment. The method for transmitting and receiving video data on the basis of a communication environment in a client comprises the steps of: driving a vision application; transmitting dummy data to a first server; receiving return dummy data from the first server; analyzing the communication environment via the return dummy data; and transmitting video data to a second server on the basis of the analyzed communication environment.

Description

Method, device, and computer program for transmitting and receiving video data based on a communication environment

[0001] The present invention relates to a method, apparatus, and computer program for transmitting and receiving image data based on a communication environment, and more specifically, to a method, apparatus, and computer program for transmitting and receiving image data more efficiently by changing the processing and transmission methods of image data according to the communication environment.

[0002] Recently, various vision-based services that utilize images input from vision devices, such as cameras, in real time are being provided in the form of applications or robot systems. Representative examples include real-time applications or robot services that provide functions such as vision-based map creation or location recognition.

[0003] Vision-based applications can be classified into edge-based and cloud-based services based on the entity performing computations on image data. In the case of edge-based services, computations on image data are performed at user terminals (clients) equipped with vision devices. In the case of cloud-based services, computations on image data are performed on a server that receives the image data from the user terminal.

[0004] When providing cloud-based services, video data must be transmitted from the user terminal to the server for computation, and the processed video data must then be transmitted back from the server to the user terminal. Therefore, the quality of the service can be influenced by the communication environment in which video data is transmitted and received. In other words, if the communication environment is poor, the time required to transmit and receive video data increases, and response times become delayed, which can have a significant impact on the overall service quality.

[0005] In other words, regarding the operation of cloud-based real-time vision applications, although there is a need to continuously monitor the communication environment and implement appropriate measures to respond when the environment is poor, no suitable solution or proposal has yet been presented.

[0006] The present invention was devised to solve the problems of the prior art as described above, and aims to provide a method, device, and computer program for transmitting and receiving image data based on a communication environment.

[0007] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program that can transmit and receive image data more efficiently by changing the image data processing and transmission methods according to the communication environment.

[0008] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program capable of presenting an image data transmission and reception method that considers the communication environment in a cloud-based vision application.

[0009] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program capable of determining the communication environment by measuring the total time from a client's image data computation request to the computationed image data response.

[0010] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program that can improve response speed while minimizing service quality degradation in real-time vision applications.

[0011] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program capable of adaptively adjusting communication capacity to match a measured communication environment.

[0012] In addition, the present invention aims to provide a communication environment-based image data transmission and reception method, device, and computer program capable of optimizing communication environment measurement data (dummy data or return dummy data) that corresponds to each image data specification.

[0013] The technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the contents described in this specification.

[0014] In a first aspect of the present invention, a method for transmitting and receiving image data based on a communication environment in a client may include: operating a vision application; transmitting dummy data to a first server; receiving return dummy data from the first server; analyzing a communication environment through the return dummy data; and transmitting image data to a second server based on the analyzed communication environment.

[0015] Here, the dummy data may be data that matches the image specifications and communication protocol of the image data.

[0016] Here, the first server is a communication environment measurement server, and the second server may be a computation server.

[0017] Here, the return dummy data includes information about the communication environment identified by the first server through the dummy data, and may be data that matches the image specifications and communication protocol of the image data.

[0018] Here, the step of transmitting video data to a second server based on the analyzed communication environment may include: a step of adjusting the video data through at least one of video size adjustment and video data compression; and a step of transmitting the adjusted video data to the second server.

[0019] In a second aspect of the present invention, a method for transmitting and receiving video data based on a communication environment in a server may include: a first server receiving dummy data from a client; a first server generating return dummy data based on the dummy data; a first server transmitting the return dummy data to the client; a second server receiving video data from the client; a second server performing computational processing on the video data; and a second server transmitting the computationally processed video data to the client.

[0020] Here, the dummy data may be data that matches the image specifications and communication protocol of the image data.

[0021] Here, the first server is a communication environment measurement server, and the second server may be a computation server.

[0022] Here, the return dummy data includes information about the communication environment identified by the first server through the dummy data, and may be data that matches the image specifications and communication protocol of the image data.

[0023] Here, the step of the second server performing computational processing on the image data may include: a step of performing decompression if the image data is compressed; a step of performing at least one of upsampling and super-resolution if the image data is resized; and a step of performing computational processing on the image data.

[0024] In a third aspect of the present invention, it may be a computer program stored on a medium to execute a method for transmitting and receiving video data based on a communication environment in a client, combined with hardware.

[0025] In a fourth aspect of the present invention, it may be a computer program stored on a medium to execute a method for transmitting and receiving image data based on a communication environment in a server, combined with hardware.

[0026] In a fifth aspect of the present invention, a client device comprising a processor and performing a method for transmitting and receiving image data based on a communication environment may perform the following steps: the processor drives a vision application; transmits dummy data to a first server; receives return dummy data from the first server; analyzes the communication environment through the return dummy data; and transmits image data to a second server based on the analyzed communication environment.

[0027] Here, the dummy data may be data that matches the image specifications and communication protocol of the image data.

[0028] Here, the first server is a communication environment measurement server, and the second server may be a computation server.

[0029] Here, the return dummy data includes information about the communication environment identified by the first server through the dummy data, and may be data that matches the image specifications and communication protocol of the image data.

[0030] Here, transmitting video data to a second server based on the analyzed communication environment may include adjusting the video data through at least one of video size adjustment and video data compression; and transmitting the adjusted video data to the second server.

[0031] In a sixth aspect of the present invention, a server device comprising a processor and performing a method for transmitting and receiving image data based on a communication environment may perform the following steps: the processor may perform the following: a first server receiving dummy data from a client; the first server generating return dummy data based on the dummy data; the first server transmitting the return dummy data to the client; a second server receiving image data from the client; the second server performing computational processing on the image data; and the second server transmitting the computationally processed image data to the client.

[0032] Here, the dummy data may be data that matches the image specifications and communication protocol of the image data.

[0033] Here, the first server is a communication environment measurement server, and the second server may be a computation server.

[0034] Here, the return dummy data includes information about the communication environment identified by the first server through the dummy data, and may be data that matches the image specifications and communication protocol of the image data.

[0035] Here, the second server performing computational processing on the image data may include: performing decompression if the image data is compressed; performing at least one of upsampling and super-resolution if image size adjustment is performed on the image data; and performing computational processing on the image data.

[0036] Accordingly, in the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention, image data can be transmitted and received more efficiently by changing the processing and transmission methods of image data according to the communication environment.

[0037] In addition, the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention may provide an image data transmission and reception method that considers the communication environment in a cloud-based vision application.

[0038] In addition, in the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention, the communication environment can be determined by measuring the total time from the client's image data operation request to the operation of the image data response.

[0039] In addition, the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention can improve response speed while minimizing degradation of service quality in real-time vision applications.

[0040] In addition, in the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention, the communication capacity can be adaptively adjusted to match the measured communication environment.

[0041] In addition, in the communication environment-based image data transmission and reception method, device, and computer program according to one embodiment of the present invention, communication environment measurement data (dummy data or return dummy data) corresponding to each image data specification can be optimized.

[0042] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present invention belongs from the contents described in this specification.

[0043] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and explain the technical concept of the present invention together with the detailed description.

[0044] FIG. 1 is a schematic diagram showing an image data transmission and reception system according to the prior art.

[0045] FIG. 2 is a schematic diagram illustrating a communication environment-based image data transmission and reception system according to one embodiment of the present invention.

[0046] FIG. 3 is a signal flow diagram illustrating a method for transmitting and receiving video data based on a communication environment between a client, a communication environment measurement server, and a computation server according to an embodiment of the present invention.

[0047] FIG. 4 is a flowchart illustrating a method for transmitting and receiving video data based on a communication environment in a client according to an embodiment of the present invention.

[0048] FIG. 5 is a flowchart illustrating a method for transmitting and receiving video data based on a communication environment in a server according to an embodiment of the present invention.

[0049] FIG. 6 illustrates an apparatus to which the proposed method of the present invention can be applied.

[0050] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings. The objects, specific advantages, and novel features of the present invention will become more apparent from the following detailed description and preferred embodiments in conjunction with the accompanying drawings.

[0051] Prior to this, the terms and words used in this specification and claims are appropriately defined by the inventor to best describe his invention and should be interpreted in a meaning and concept consistent with the technical spirit of the invention; they are intended merely to describe embodiments and should not be interpreted as limiting the invention.

[0052] In assigning reference numerals to components, identical or similar components are assigned the same reference numeral regardless of the reference numeral, and redundant descriptions thereof are omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably for the sake of ease of drafting the specification; they do not inherently possess distinct meanings or roles and may refer to software or hardware components.

[0053] In describing the components of the present invention, when a component is expressed in a singular form, it should be understood that the component also includes a plural form unless specifically stated otherwise. Furthermore, terms such as "first," "second," etc., are used to distinguish one component from another, and the components are not limited by said terms. Additionally, if one component is connected to another component, it implies that yet another component may be connected between the components.

[0054] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art may obscure the essence of the embodiments disclosed in this specification, such detailed description is omitted. Furthermore, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that the drawings include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the invention.

[0055] Hereinafter, exemplary embodiments of a communication environment-based image data transmission and reception method, apparatus, and computer program according to the present invention will be described in detail with reference to the accompanying drawings.

[0056] FIG. 1 is a schematic diagram showing an image data transmission and reception system according to the prior art.

[0057] Referring to FIG. 1, a video data transmission and reception system according to the prior art may include a client (terminal device) and a server (computation server).

[0058] Clients and servers can be connected via a network (Internet). The network may include wired networks and wireless networks, and specifically, may include various networks such as Local Area Networks (LAN), Metropolitan Area Networks (MAN), and Wide Area Networks (WAN). Additionally, the network may include the known World Wide Web (WWW). However, the network according to one embodiment of the present invention is not limited to the networks listed above and may include at least a known wireless data network, a known telephone network, or a known wired or wireless television network.

[0059] The client can run various vision-based applications that utilize images input from an onboard vision device (e.g., a camera) in real time. These real-time applications may include, for example, applications for vision-based map creation or location recognition.

[0060] Vision-based applications running on the client can be classified into edge-based and cloud-based applications depending on where the computation on the image data takes place. Edge-based applications perform computations on image data using a processor installed on the client, whereas cloud-based applications perform computations on image data on a server that receives the data from the client.

[0061] To process video data in cloud-based applications, the client must transmit the video data to a computation server, the server must process it, and then transmit the processed video data back to the client. However, if the communication environment is poor and communication is not smooth, the time required to transmit and receive video data may increase or response times may be delayed, which can degrade overall performance; in some cases, the video data transmission itself may even fail.

[0062] Although a smartphone is illustrated as an example of a client in FIG. 1, a client according to one embodiment of the present invention may include various devices such as a desktop computer, a laptop computer, a PDA (Personal Digital Assistant), a web pad, and a tablet PC that can connect to a network and communicate with a computing server, in addition to a smartphone.

[0063] A client can run a web browser and load a web page on the web browser according to user input. The client may include a display unit for displaying the loaded web page to a user, an input unit for receiving user input, memory storing at least one program, and a processor. Here, at least one program may include a web browser and other components associated with the operation of the web browser.

[0064] The computation server can store computationally processed image data and various information that can be provided to a client, transmits the computationally processed image data and various information to the client upon the client's request, and can provide web services based on this data and information.

[0065] FIG. 2 is a schematic diagram illustrating a communication environment-based image data transmission and reception system according to one embodiment of the present invention.

[0066] Referring to FIG. 2, a communication environment measurement server for measuring the communication environment is provided on the server side along with an existing computation server. The communication environment measurement server is deployed on the same network as the computation server that performs image data computation or processing algorithms, enabling the measurement of the communication speed (communication environment) between the client and the computation server.

[0067] The client communicates with a communication environment measurement server at the beginning of vision application operation or periodically to determine the communication environment between the client and the computation server. This communication for determining the communication environment can be executed in the background.

[0068] When a client transmits dummy data for measuring the communication environment to a communication environment measurement server, the communication environment measurement server receives it and generates return dummy data by adding only information about the communication environment to the dummy data without separate computational processing on the dummy data, and transmits it to the client (or it may generate return dummy data using only information about the communication environment and transmit it to the client).

[0069] Upon receiving return dummy data containing information about the communication environment from the communication environment measurement server, the client identifies the communication environment and determines whether to adjust the video data accordingly. Then, based on this decision, the client transmits the adjusted or unadjusted video data to the server-side processing server so that the video data can be processed.

[0070] When the computation server receives adjusted image data, if the adjusted image data is image data with adjusted size, the computation server may up-sample or super-resolution process the adjusted image data, perform computation processing on the image data, and transmit the processed image data to the client side. When the computation server receives adjusted image data, if the adjusted image data is compressed image data, the computation server may decompress (decode) the adjusted image data, perform computation processing on the image data, and transmit the processed image data to the client side. When the computation server receives unadjusted image data, it may perform computation processing on the image data and transmit the processed image data to the client side.

[0071] FIG. 3 is a signal flow diagram illustrating a method for transmitting and receiving video data based on a communication environment between a client, a communication environment measurement server, and a computation server according to an embodiment of the present invention.

[0072] Referring to FIG. 3, when a cloud-based real-time vision application is launched on a client, it checks whether dummy data matching the image specifications of the target image data is stored on the client. If the check reveals that there is no dummy data matching the specifications of the target image data, the client generates dummy data. Here, the dummy data is data that matches each image specification and conforms to the communication protocol; by caching the dummy data in memory in advance on the client, unnecessary computations can be prevented. The client then checks whether the time for measuring the communication environment has arrived, and if it has, transmits the dummy data to the communication environment measurement server. Here, the time for measuring the communication environment may be the start time of the vision application launch or a regularly scheduled time. Upon receiving the dummy data, the communication environment measurement server identifies the communication environment through the received dummy data and generates return dummy data containing information about the identified communication environment. It then transmits this to the client. At this time, the communication environment identified through the dummy data (or return dummy data) can be determined through indicators such as the transmission and reception time of the dummy data (or return dummy data) and the error rate of the dummy data (or return dummy data). Furthermore, the return dummy data is data that matches each video specification and conforms to the return communication protocol, and the communication environment measurement server may prevent unnecessary calculations by caching the return dummy data in memory in advance. Subsequently, the client receives the return dummy data from the communication environment measurement server and analyzes the communication environment through the received return dummy data. The client can identify the communication environment through indicators such as the transmission and reception time of the return dummy data and the error rate of the return dummy data. Subsequently, based on the identified communication environment, the client checks whether adjustment of the video data is necessary.If it is determined that video adjustment is necessary, the video data is adjusted through video size adjustment or video data compression. If it is determined that video adjustment is not necessary, the unadjusted video data is transmitted to the processing server. If it is determined that video adjustment is necessary and the video data is adjusted through video data compression, the adjusted video data and compression encoding information are transmitted to the processing server. If it is determined that video adjustment is necessary and the video data is adjusted solely through video size adjustment, the adjusted video data is transmitted to the processing server. Whether video data adjustment is required and the method of adjustment (e.g., video size adjustment method or video data compression method) can be determined by the settings. For example, if a longer time is taken than the configured transmission and reception time, it may be determined that video data adjustment is required. For example, let Dp be the maximum allowable time delay for processing a single image between the client and the server according to a real-time vision application, Dt be the maximum allowable time delay for transmitting and receiving image data, Tt be the time taken from the client sending dummy data to receiving return dummy data from the server, and Tr be the sum of the time taken from the client sending dummy data to the server and the time taken from the server sending return dummy data to the client. Then Dt <Tt-Dp인 경우(또는 Dt<Tr인 경우) 영상 데이터의 조정이 필요하다고 판단될 수 있다. 영상 조정이 필요하다고 판단된 경우, 영상 사이즈 조절이나 영상 데이터 압축을 통해 영상 데이터를 조정할 수 있다. 즉, 영상 데이터 조정은 영상 사이즈 조절과 영상 데이터 압축을 다양하게 조합할 수 있다.For example, various combinations are possible, such as JPEG compression at 50% for the original size, 80% for the original size, 50% for half size, 80% for half size, 50% for one-third size, and 80% for one-third size. If it is determined that video adjustment is necessary and the video data is adjusted through compression, the client transmits the adjusted video data and compression encoding information to the computing server. If it is determined that video adjustment is necessary and the video data is adjusted solely by changing the video size without compression, the client transmits the adjusted video data to the computing server. Upon receiving video data from the client, the computing server checks whether the received video data is the adjusted video data. The computing server then determines whether video enhancement is required for the adjusted video data. Whether video enhancement is required can be determined by settings (for example, it can be determined based on the degree of image quality degradation). If it is determined that image enhancement is required, the computation server checks whether image data compression has been performed on the adjusted image data; if image data compression has been performed on the adjusted image data, it performs decoding (decompression) that matches the compression encoding information. Subsequently, the computation server checks whether image size adjustment has been performed on the adjusted image data; if image size adjustment has been performed on the adjusted image data, it performs upsampling. Then, if the quality of the enhanced image is lower than a predetermined threshold (Qt) after upsampling, it performs super-resolution again. Subsequently, the computation server performs computation processing on the image data and transmits the processed image data to the client.

[0073] FIG. 4 is a flowchart illustrating a method for transmitting and receiving video data based on a communication environment in a client according to an embodiment of the present invention.

[0074] A method (400) for transmitting and receiving image data based on a communication environment in a client according to an embodiment of the present invention comprises: a step of running a cloud-based real-time vision application (401); a step of checking whether there is dummy data that matches the image specifications of the image data to be transmitted (403); a step of generating dummy data if there is no dummy data that matches the image specifications of the image data to be transmitted (405); a step of checking whether the time for measuring the communication environment has arrived (407); a step of transmitting dummy data if the time for measuring the communication environment has arrived (409); a step of receiving return dummy data (411); a step of analyzing the communication environment through the received return dummy data (413); a step of checking whether adjustment of the image data is required (415); a step of adjusting the image data by adjusting the image size or compressing the image data if it is determined that adjustment of the image is required (417); a step of transmitting the image data if it is determined that adjustment of the image is not required (419); a step of transmitting the adjusted image data and compression encoding information if it is determined that adjustment of the image is required and the image data is adjusted through compression of the image data (421); and a step of transmitting the adjusted image data and compression encoding information (423). If it is determined that video adjustment is necessary and the video data is adjusted only by adjusting the video size, the method may include the step of transmitting the adjusted video data (425) and the step of receiving the video data processed by the server (427).

[0075] In the step (403) of checking whether there is dummy data that matches the image specifications of the image data to be transmitted, the dummy data is data that matches each image specification (image specifications of each image data) and conforms to the communication protocol for transmitting and receiving image data, and the client may cache the dummy data in memory in advance to prevent unnecessary operations from being performed.

[0076] In the step (407) of checking whether the time for measuring the communication environment has arrived, the time for measuring the communication environment may be the time when the vision application starts running, or a time set periodically.

[0077] When the time for measuring the communication environment arrives, in the step of transmitting dummy data (409), the client transmits the dummy data to the communication environment measurement server.

[0078] In the step (411) of receiving return dummy data, the client receives return dummy data from the communication environment measurement server. The communication environment measurement server identifies the communication environment through the dummy data received from the client, generates return dummy data containing information about the identified communication environment, and transmits it to the client. At this time, the communication environment identified through the dummy data (or return dummy data) can be identified through indicators such as the time required for transmission and reception of the dummy data (or return dummy data) and the error rate of the dummy data (or return dummy data). Additionally, the return dummy data is data that matches each video specification (video specification of each video data) and conforms to the return communication protocol for video data transmission and reception, and the server may cache the return dummy data in memory in advance to prevent unnecessary calculations from being performed.

[0079] In the step (413) of analyzing the communication environment through the received return dummy data, the client can identify the communication environment through indicators such as the time taken to send and receive the return dummy data and the error rate of the return dummy data.

[0080] In the step (415) of checking whether adjustment of the image data is required, whether adjustment of the image data is required can be determined by the settings. For example, if a longer time is taken than the set transmission and reception time, it may be determined that adjustment of the image data is required. For instance, let Dp be the maximum time delay allowed for processing one image between the client and the server according to a real-time vision application, Dt be the maximum time delay allowed for transmitting and receiving image data, Tt be the time taken from the client to transmit dummy data and to receive return dummy data from the server, and Tr be the sum of the time taken from the client to transmit dummy data to the server and the time taken from the server to transmit return dummy data to the client. Then Dt <Tt-Dp인 경우(또는 Dt<Tr인 경우) 영상 데이터의 조정이 필요하다고 판단될 수 있다.

[0081] When it is determined that image adjustment is necessary, in the step (417) of adjusting image data through image size adjustment or image data compression, the image data adjustment can be performed through image size adjustment or image data compression. Therefore, image data adjustment can be various combinations of image size adjustment and image data compression. For example, various combinations are possible, such as JPEG compression rate of 50% for the original size, JPEG compression rate of 80% for the original size, JPEG compression rate of 50% for half size, JPEG compression rate of 80% for half size, JPEG compression rate of 50% for one-third size, JPEG compression rate of 80% for one-third size, JPEG compression rate of 80% for one-third size, etc.

[0082] When it is determined that video adjustment is necessary and the video data is adjusted through video data compression, in the step (423) of transmitting the adjusted video data and compression encoding information, if the client has adjusted the video data through video data compression, the compression encoding information must be transmitted along with the adjusted video data so that the server can decode it.

[0083] FIG. 5 is a flowchart illustrating a method for transmitting and receiving video data based on a communication environment in a server according to an embodiment of the present invention.

[0084] A server according to one embodiment of the present invention may include a communication environment measurement server for measuring a communication environment and a computation server for processing image data.

[0085] A method (500) for transmitting and receiving video data based on a communication environment in a server according to an embodiment of the present invention comprises: a step (501) in which a communication environment measurement server (first server) receives dummy data from a client; a step (503) in which a communication environment measurement server generates return dummy data; a step (505) in which a communication environment measurement server transmits return dummy data to a client; a step (507) in which a computation server (second server) receives video data from a client; a step (509) in which a computation server checks whether adjusted video data has been received from a client; a step (511) in which a computation server determines whether video improvement is required for the adjusted video data; a step (513) in which a computation server checks whether video data compression has been performed on the adjusted video data; a step (515) in which, if video data compression has been performed on the adjusted video data, decoding (decompression) matching the compression encoding information is performed; a step (517) in which a computation server checks whether video size adjustment has been performed on the adjusted video data; and a step (519) in which, if video size adjustment has been performed on the adjusted video data. The method may include a step (521) of performing super-resolution when the quality of the improved image after up-sampling is lower than a predetermined threshold (Qt), a step (525) of a computation server performing computation processing on the image data, and a step (527) of the computation server transmitting the processed image data to a client.

[0086] In the step (501) where the communication environment measurement server (first server) receives dummy data from the client, the dummy data is data that matches each video specification and conforms to the communication protocol, and the client may cache the dummy data in memory in advance to prevent unnecessary calculations from being performed.

[0087] In the step (503) where the communication environment measurement server generates return dummy data, the communication environment measurement server identifies the communication environment through the dummy data received from the client, generates return dummy data containing information about the identified communication environment, and transmits it to the client. At this time, the communication environment identified through the dummy data (or return dummy data) can be identified through indicators such as the time required for transmission and reception of the dummy data (or return dummy data) and the error rate of the dummy data (or return dummy data). Additionally, the return dummy data is data that matches each video specification and conforms to the return communication protocol, and the communication environment measurement server may cache the return dummy data in memory in advance to prevent unnecessary calculations from being performed.

[0088] In the step (511) where the computation server determines whether image improvement is necessary for the adjusted image data, if the computation server receives adjusted image data in which the image size has been adjusted or the image data has been compressed, it determines whether to improve the degradation that has occurred in the image. In the case where the image size has been adjusted, the size can be determined immediately through the image data received by the computation server. However, in the case of image data compression, the client must separately transmit compression encoding information to the computation server so that the details regarding image data compression can be determined.

[0089] When the image size is adjusted, a process to restore it is required on the computing server. For example, if the image size is reduced by half, it must be up-sampled by a factor of two. In this case, since simply performing up-sampling can result in detectable image quality degradation, super-resolution can be applied to improve the image. Deep learning-based technologies such as FSRCNN, DRCT-L, HMA, and HAT-L can be used for super-resolution.

[0090] When video data is compressed, decoding (decompression) matching the video data compression method (compression encoding information) must be applied (it is necessary to adopt a decoding method corresponding to the compression encoding method). However, even in the case of data loss methods such as JPEG compression, there may be instances where the degradation in image quality caused by compression is accepted without performing decoding or restoration. Both hand-crafted compression methods such as JPEG, VTM, and BPG, and deep learning-based compression methods can be used for compression.

[0091] A device to which the proposed method of the present invention can be applied

[0092] FIG. 6 illustrates a device (600) to which the proposed method of the present invention can be applied. The device (600) may include a server or terminal that transmits and receives video data based on a communication environment.

[0093] Referring to FIG. 6, the device (600) may be a server device or a terminal device configured to implement a process for transmitting and receiving video data based on a communication environment.

[0094] For example, a device (600) to which the proposed method of the present invention can be applied may include network devices such as repeaters, hubs, bridges, switches, routers, gateways, etc., computer devices such as desktop computers, workstations, etc., mobile terminals such as smartphones, portable devices such as laptop computers, home appliances such as digital TVs, means of transportation such as automobiles, etc. As another example, a device (600) to which the present invention can be applied may be included as part of an Application Specific Integrated Circuit (ASIC) implemented in the form of a System On Chip (SoC).

[0095] The memory (620) can be connected to the processor (610) during operation and can store programs and / or instructions for processing and controlling the processor (610), and can store data and information used in the present invention, control information required for data and information processing according to the present invention, temporary data generated during the data and information processing process, etc. The memory (620) can be implemented as a storage device such as ROM (Read Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, SRAM (Static RAM), HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0096] The processor (610) can be operatively connected to the memory (620) and network interface (630) and controls the operation of each module within the device (600). In particular, the processor (610) can perform various control functions to carry out the proposed method of the present invention. The processor (620) may also be referred to as a controller, microcontroller, microprocessor, microcomputer, etc. The proposed method of the present invention may be implemented by hardware, firmware, software, or a combination thereof. When implementing the present invention using hardware, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. configured to carry out the present invention may be provided in the processor (1010). Meanwhile, when implementing the proposed method of the present invention using firmware or software, the firmware or software may include instructions related to modules, procedures, or functions that perform functions or operations necessary to implement the proposed method of the present invention, and the instructions may be stored in memory (620) or stored in a computer-readable recording medium (not shown) separately from memory (620) and configured so that the device (600) implements the proposed method of the present invention when executed by the processor (610).

[0097] Additionally, the device (600) may include a network interface device (630). The network interface device (630) is connected to the processor (610) during operation, and the processor (610) can control the network interface device (630) to transmit or receive wireless / wired signals carrying information and / or data, signals, messages, etc. through a wireless / wired network. The network interface device (630) supports various communication standards, such as the IEEE 802 series, 3GPP LTE(-A), 3GPP 5G, etc., and can transmit and receive control information and / or data signals according to the communication standards. The network interface device (630) may be implemented outside the device (600) as needed.

[0098] The embodiments and drawings described herein are merely illustrative and do not limit the scope of the invention in any way. Furthermore, the connections of lines or connecting members between the components depicted in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in the actual device. Additionally, unless specifically stated with terms such as "essential" or "importantly," a component may not be strictly necessary for the application of the invention.

[0099] In the specification of the present invention (particularly in the claims), the use of the term "the above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is implied to include inventions applying individual values ​​belonging to said range (unless otherwise stated), and is equivalent to describing each individual value constituting said range in the detailed description of the invention. Additionally, the steps presented in the method inventions of the present invention are not intended to impose a constraint on their sequential order, and the order may be appropriately changed as necessary, unless a specific step must necessarily precede another step depending on the nature of each process. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by such examples or exemplary terms unless limited by the claims. Furthermore, a person skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and elements within the scope of the claims or equivalents.

Claims

1. In a method for transmitting and receiving video data based on a communication environment at a client, Step of running a vision application; A step of transmitting dummy data to the first server; A step of receiving return dummy data from the first server; A step of analyzing the communication environment through the above-mentioned return dummy data; and A method comprising the step of transmitting video data to a second server based on the analyzed communication environment.

2. In Paragraph 1, A method in which the above dummy data is data that matches the image specifications and communication protocol of the above image data.

3. In Paragraph 1, The first server is a communication environment measurement server, and The second server is a computational server, method.

4. In Paragraph 1, The above return dummy data is, The first server includes information about the communication environment identified through the above dummy data, and A method, which is data matching the image specifications and communication protocol of the above-mentioned image data.

5. In Paragraph 1, The step of transmitting video data to a second server based on the analyzed communication environment is: A step of adjusting the image data through at least one of image size adjustment and image data compression; and A method comprising the step of transmitting the above-mentioned adjusted image data to a second server.

6. In a method for transmitting and receiving video data based on a communication environment on a server, A step in which the first server receives dummy data from the client; A step in which the first server generates return dummy data based on the above dummy data; A step in which the first server transmits the return dummy data to the client; A step in which the second server receives video data from the client; A step in which a second server performs computational processing on the image data; and A method comprising the step of a second server transmitting the processed image data to the client.

7. In Paragraph 6, A method in which the above dummy data is data that matches the image specifications and communication protocol of the above image data.

8. In Paragraph 6, The first server is a communication environment measurement server, and The second server is a computational server, method.

9. In Paragraph 6, The above return dummy data is, The first server includes information about the communication environment identified through the above dummy data, and A method, which is data matching the image specifications and communication protocol of the above-mentioned image data.

10. In Paragraph 6, The step of the second server performing computational processing on the image data is: A step of performing decompression when the above image data is compressed; If image size adjustment is performed on the above image data, a step of performing at least one of upsampling and super-resolution; and A method comprising the step of performing computational processing on the above image data.

11. A client device comprising a processor and performing a method for transmitting and receiving image data based on a communication environment, The above processor is, Running a vision application; Sending dummy data to the first server; Receiving return dummy data from the first server; Analyzing the communication environment through the above-mentioned return dummy data; and A device that performs the following: transmitting video data to a second server based on the analyzed communication environment.

12. In Paragraph 11, A device in which the above dummy data is data that matches the image specifications and communication protocol of the above image data.

13. In Paragraph 11, The first server is a communication environment measurement server, and The second server is a device that is a computation server.

14. In Paragraph 11, The above return dummy data is, The first server includes information about the communication environment identified through the above dummy data, and A device that is data matching the image specifications and communication protocol of the above image data.

15. In Paragraph 11, Transmitting video data to a second server based on the analyzed communication environment is, Adjusting the image data through at least one of image size adjustment and image data compression; and A device comprising transmitting the above-mentioned adjusted image data to a second server.

16. A server device comprising a processor and performing a method for transmitting and receiving image data based on a communication environment, The above processor is, The first server receiving dummy data from the client; The first server generates return dummy data based on the above dummy data; The first server transmits the return dummy data to the client; The second server receiving video data from the above client; The second server performs computational processing on the above image data; and A device that executes, including the second server transmitting the processed image data to the client.

17. In Paragraph 16, A device in which the above dummy data is data that matches the image specifications and communication protocol of the above image data.

18. In Paragraph 16, The first server is a communication environment measurement server, and The second server is a device that is a computation server.

19. In Paragraph 16, The above return dummy data is, The first server includes information about the communication environment identified through the above dummy data, and A device that is data matching the image specifications and communication protocol of the above image data.

20. In Paragraph 16, The second server performing computational processing on the above image data is, If the above image data is compressed, performing decompression; If image size adjustment is performed on the above image data, at least one of upsampling and super-resolution is performed; and A device comprising: performing computational processing on the above image data.

Images