Transmission device, transmission system, transmission method, and program
The transmission system addresses the challenge of balancing video signal processing delays by using an adjustment unit to select optimal signal processing combinations, ensuring the system meets allowable delay limits, thus enhancing real-time video communication performance and quality.
Patent Information
- Application Number
- PCT/JP2023/020151
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-05
AI Technical Summary
Existing video transmission systems face challenges in selecting appropriate signal processing methods that balance transmission delay and processing delay within a predetermined allowable delay, crucial for maintaining real-time performance in video communication.
A transmission system that includes an adjustment unit to select combinations of signal processing algorithms from multiple options, ensuring the sum of transmission delay and total processing delay remains within a user-defined allowable limit, while applying these selected processes to video signals to optimize delay and quality.
Enables video communication with reduced processing delay and maintained high quality, allowing users to perform real-time video communication without discomfort by selecting signal processing combinations that meet the predetermined delay constraints.
Smart Images

Figure JP2023020151_05122024_PF_FP_ABST
Abstract
Description
Transmitting device, transmission system, transmitting method, and program
[0001] The present invention relates to a transmitting device, a transmission system, a transmitting method, and a program.
[0002] In a transmission system for transmitting and receiving video signals, a video signal acquired by a transmitting device undergoes various signal processing, and is then transmitted to a receiving device via a network or the like.
[0003] Known methods for applying signal processing to a video signal include those disclosed in Non-Patent Documents 1 and 2. A receiving device receives the video signal and displays it on a display or the like to present the video to a user. This enables video communication between a user on the transmitting device side and a user on the receiving device side.
[0004] J. Lawrenceet al., “Project Starline: A High-Fidelity Telepresence System,” ACM Trans. Graph.,vol. 40, no. 6, Dec. 2021, doi: 10.1145 / 3478513.3480490.Y. Zhanget al., “VirtualCube: An Immersive 3D Video Communication System,” in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 5, pp. 2146-2156, May 2022, doi: 10.1109 / TVCG.2022.3150512.
[0005] Transmission delays occur when transmitting video signals. Furthermore, in video communication, there is an allowable delay (hereinafter referred to as "tolerable delay"). The allowable delay differs depending on the use case of the video communication. For example, in video communication where real-time performance is important, the allowable delay is set short. In such cases, it is desirable to apply signal processing to the video signal that minimizes transmission delay.
[0006] However, the techniques disclosed in the above-mentioned non-patent documents 1 and 2 have a problem in that the signal processing to be applied to the video signal cannot be selected based on the relationship between the total processing delay of each signal processing (the delay required when performing each signal processing on data) and the allowable delay.
[0007] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide a transmitting device, a transmission system, a transmitting method, and a program that are capable of selecting appropriate signal processing according to a specified allowable delay.
[0008] A transmitting device according to one aspect of the present disclosure is a transmitting device that adds any signal processing selected from a plurality of signal processing methods to a video signal and transmits the signal, and includes: an adjustment unit that selects one or more combinations of signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay; a signal processing unit that adds the signal processing selected by the adjustment unit to the video signal; and a transmitting unit that transmits the signal-processed video signal.
[0009] A transmission system according to one aspect of the present disclosure is a transmission system comprising a transmitting device that adds any signal processing selected from a plurality of signal processing methods to a video signal and transmits the signal, and a receiving device that receives the transmitted video signal, wherein the transmitting device comprises an adjustment unit that selects one or more combinations of signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay, a signal processing unit that adds the signal processing selected by the adjustment unit to the video signal, and a transmitting unit that transmits the signal-processed video signal, and the receiving device comprises a receiving unit that receives the transmitted video signal and a display unit that displays the received video signal.
[0010] A transmission method according to one aspect of the present disclosure is a transmission method in which any signal processing selected from a plurality of signal processing methods is added to a video signal and the signal is transmitted, wherein one or a combination of multiple signal processing methods is selected from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay, the selected signal processing is added to the video signal, and the signal-processed video signal is transmitted.
[0011] One aspect of the present disclosure is a program for causing a computer to function as the transmission device.
[0012] According to the present disclosure, it becomes possible to select appropriate signal processing depending on a predetermined allowable delay.
[0013] FIG. 1 is a block diagram showing the configuration of a transmission system according to an embodiment. FIG. 2 is an explanatory diagram showing processing delay and quality of signal processing. FIG. 3 is an explanatory diagram showing a signal processing combination, total processing delay, and total quality. FIG. 4 is a flowchart showing the processing procedure of a transmission system according to an embodiment. FIG. 5 is a flowchart showing the processing procedure of a signal processing combination determination process of a transmission system according to a first embodiment. FIG. 6 is a flowchart showing the processing procedure of a signal processing combination determination process of a transmission system according to a second embodiment. FIG. 7 is a flowchart showing the processing procedure of a signal processing combination determination process of a transmission system according to a third embodiment. FIG. 8 is a block diagram showing the hardware configuration of this embodiment.
[0014] [First Embodiment] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a transmission system 100 according to an embodiment. As shown in Fig. 1, the transmission system 100 includes a transmitting device 1, a receiving device 2, and a measuring unit 3. The transmitting device 1 and the receiving device 2 are connected via a network, for example. The transmitting device 1 applies any of a plurality of signal processing processes to a video signal and transmits the signal.
[0015] 1 illustrates an example in which a video signal is transmitted from a transmitting device 1 installed in one base station TA1 to a receiving device 2 installed in another base station TA2. In addition to the configuration illustrated in FIG. 1, it is also possible to configure one base station TA1 to be equipped with a receiving device 2 and the other base station TA2 to be equipped with a transmitting device 1, thereby transmitting and receiving video signals bidirectionally between the one base station TA1 and the other base station TA2. Hereinafter, the sum of the transmission processing delay required for processing from when the video is acquired by the transmitting device 1 to when the video is displayed by the receiving device 2 and the propagation delay required for propagation will be referred to as "transmission delay." Furthermore, the delay required for each signal processing will be referred to as "processing delay."
[0016] The measurement unit 3 measures the transmission delay and the total processing delay when a video signal is transmitted from the transmitting device 1 to the receiving device 2. The measurement unit 3 may be installed in either the transmitting device 1 or the receiving device 2. The measurement unit 3 may be connected to the transmitting device 1 and the receiving device 2 via a network.
[0017] The transmitting device 1 includes an acquisition unit 11 , a signal processing unit 12 , an adjustment unit 13 , and a transmitting unit 14 .
[0018] The acquisition unit 11 includes an imaging device such as an RGB camera and a depth camera. The acquisition unit 11 acquires a video signal and depth information detected by the imaging device. The acquisition unit 11 generates a video signal including depth information such as point cloud data and RGBD data based on the acquired video signal and depth information. The acquisition unit 11 outputs the generated video signal to the signal processing unit 12.
[0019] The signal processing unit 12 has a plurality of signal processing algorithms. The signal processing includes processing for removing noise from the video signal, processing for increasing the resolution of the video signal, processing for realizing eye contact between users, processing for applying lighting when displaying people included in the video signal, three-dimensional face tracking processing, etc. The signal processing unit 12 applies desired signal processing to the video signal output from the acquisition unit 11.
[0020] For each of the above signal processing methods, there are multiple types of signal processing of the same type. For example, there are multiple signal processing methods with different algorithms for the purpose of noise removal. In such cases, even if the algorithms are different, they are treated as the same type of signal processing.
[0021] 2 is a diagram showing an example of the correspondence between each signal processing, processing delay, and quality. As described above, the "transmission delay" is a value obtained by adding the transmission processing delay required for transmitting and receiving a video signal and the propagation delay, and is measured by the measurement unit 3.
[0022] As a method for measuring transmission delay, for example, a method can be adopted in which PTP (Precision Time Protocol) is adopted, time synchronization is performed between the transmitting device 1 and the receiving device 2, and the difference in arrival time of data is measured.
[0023] As shown in Fig. 2, signal processing A includes signal processing A1, A2, and A3, which are the same type but have different algorithms. Signal processing B includes signal processing B1 and B2, which are the same type but have different algorithms. Signal processing C includes signal processing C1, C2, and C3, which are the same type but have different algorithms.
[0024] The processing delay and quality of each signal processing are not the same, and differ for each signal processing. As shown in Figure 2, the processing delay t of each signal processing A1 to A3, B1, B2, and C1 to C3 is t1 to t8, respectively, and the quality q is q1 to q8, respectively. The quality q is an index expressed as a numerical value, and the higher the numerical value, the better the quality. Normally, the higher the quality, the greater the processing delay. In other words, applying high-quality signal processing to a video signal increases the processing delay, and lowering the quality of the signal processing reduces the processing delay. Note that in some cases, the higher the quality, the shorter the processing delay.
[0025] In the following, when a processing delay is specifically indicated, it will be indicated with a suffix such as "processing delay t1", and when it is not specifically indicated or when it is referred to generically, it will be indicated without a suffix such as "processing delay t". The same applies to quality q (q1 to q8).
[0026] The signal processing unit 12 applies one or more combinations of signal processing set by the adjustment unit 13 to the video signal acquired by the acquisition unit 11, and outputs the result to the transmission unit 14. That is, the signal processing unit 12 applies the signal processing selected by the adjustment unit 13 to the video signal. Hereinafter, one signal processing and a combination of multiple signal processing are collectively referred to as a "combination of signal processing." That is, a "combination of signal processing" includes not only a combination of processing multiple signals, but also one signal processing.
[0027] Based on a predetermined allowable delay set by a user and the transmission delay measured by the measurement unit 3, the adjustment unit 13 extracts a combination of signal processing in which the sum of the transmission delay and the total processing delay is equal to or less than the allowable delay.
[0028] The "allowable delay" is the tolerance value of the sum of the processing delays that occur when applying each signal processing to the video and the transmission delay (the sum of the transmission delay and the total processing delay). The allowable delay is a value that varies depending on the use case of the video communication and is initially set by the user. For example, in the case of video communication where real-time performance is important, the allowable delay is set to a short value.
[0029] FIG. 3 is a diagram showing the relationship between signal processing combinations, the total processing delay (denoted by the symbol "T") when each signal processing combination is applied, and the total quality (denoted by the symbol "Q"). The processing delay T can be represented, for example, by the sum of the individual processing delays t shown in FIG. 2. The total quality Q can be represented, for example, by the sum of the individual qualities q shown in FIG. 2. In other words, the total quality Q includes a numerical value obtained by summing the individual qualities q of each signal processing included in the selected combination. Furthermore, the processing delay T and the quality Q are estimated values.
[0030] When setting combinations of signal processing, the adjustment unit 13 avoids duplication of the same type of signal processing. Specifically, as shown in FIG. 2, signal processing A1 to A3 are the same type of signal processing (e.g., noise reduction processing), and therefore these signal processing operations are not set to be duplicated. That is, when there are multiple signal processing operations of the same type, the adjustment unit 13 selects one of the same type of signal processing operations. Therefore, when three types of signal processing A, B, and C are applied to a video signal, there will be the following combinations (referred to as n combinations) as shown in FIG. 3: A1-B1-C1, A1-B1-C2, A1-B1-C3, A1-B2-C1, ..., A3-B2-C3.
[0031] Furthermore, the adjustment unit 13 calculates the processing delay T and quality Q for each combination of signal processing by changing the execution order of the signal processing. For example, for the signal processing combination "A1-B1-C1," the adjustment unit 13 sets "A1-C1-B1," "B1-A1-C1," "B1-C1-A1," "C1-A1-B1," and "C1-B1-A1" by changing the order of the signal processing in this combination, and calculates the processing delay T and quality Q for each of these. The number of combinations of the above-mentioned signal processing is assumed to be "m." Hereinafter, a combination in which the same type of signal processing does not overlap is referred to as a "candidate solution." Therefore, in the example shown in FIG. 3, there are m candidate solutions.
[0032] The adjustment unit 13 calculates the processing delay T (T1 to Tm) for each of the candidate solutions described above. Hereinafter, when the processing delay of each candidate solution is specifically indicated, it will be indicated with a suffix such as "processing delay T1," and when it is not specifically indicated or when it is referred to collectively, it will be indicated without a suffix such as "processing delay T." The same applies to the quality Q (Q1 to Qm).
[0033] The adjustment unit 13 calculates the qualities Q1 to Qm when the signal processing of each candidate solution is applied. As a method for evaluating the quality of the signal processing, PSNR (Peak Signal to Noise Ratio), SSIM (Structural SIMilarity), or the like can be adopted. Furthermore, evaluation methods other than PSNR and SSIM may also be adopted. The signal processing qualities Q1 to Qm of each candidate solution may be a numerical value obtained by summing the quality q of each signal processing shown in FIG. 2 .
[0034] The adjustment unit 13 extracts candidate solutions whose sum of the processing delay T and the transmission delay is equal to or less than the allowable delay Tth. For example, in the example shown in FIG. 3, "A1-B1-C1," "A1-B2-C1," "A3-B2-C3," and "C3-B2-A3" satisfy the condition (T+α≦Tth). Note that "α" indicates the transmission delay. Therefore, "T+α≦Tth" indicates that the sum of the processing delay T and the transmission delay α is equal to or less than the allowable delay Tth. The adjustment unit 13 extracts candidate solutions with the highest quality Q (e.g., the top five candidate solutions) from the extracted candidate solutions.
[0035] The adjustment unit 13 acquires the actual measurement value of the quality Q of each of the top five candidate solutions, extracts the candidate solution with the highest quality Q, and outputs it to the signal processing unit 12. That is, the adjustment unit 13 determines the candidate solution (combination and order of signal processing) that results in the highest quality Q based on the allowable delay Tth and the processing delay T, and outputs it to the signal processing unit 12. The actual measurement value of the quality Q can be measured using the above-mentioned PSNR, SSIM, etc.
[0036] That is, the adjustment unit 13 has a function of selecting one or a combination of multiple signal processing methods from multiple signal processing methods so that the sum of the total processing delay T and the transmission delay is equal to or less than a predetermined allowable delay Tth. The adjustment unit 13 also has a function of selecting a combination of signal processing methods so that the total quality Q of the combined signal processing methods is the highest.
[0037] 1 shows an example in which the adjustment unit 13 is provided in the transmitting device 1, but if the transmitting device 1 does not have sufficient resources for calculation processing, the adjustment unit 13 may be provided in the receiving device 2. Furthermore, if both the transmitting device 1 and the receiving device 2 do not have sufficient resources for calculation processing, the processing of the adjustment unit 13 may be executed by a device other than the transmitting device 1 and the receiving device 2, such as a cloud.
[0038] Returning to FIG. 1 , the transmission unit 14 transmits the video signal processed by the signal processing unit 12 to the receiving device 2 .
[0039] The receiving device 2 includes a receiving unit 21 and a display unit 22 .
[0040] The receiving unit 21 receives the video signal transmitted from the transmitting device 1 .
[0041] The display unit 22 includes an image display device such as a naked-eye three-dimensional display, and displays the image signal received by the receiving unit 21 in three dimensions.
[0042] Next, the operation of the above-described embodiment will be described with reference to the flowcharts shown in Figures 4 and 5. Figure 4 is a flowchart showing the processing procedure of the transmission system 100 according to the embodiment.
[0043] First, in step S1, the adjustment unit 13 acquires the allowable delay Tth initially set by the user. As described above, the allowable delay Tth is a value that varies depending on the use case of the video communication. For example, in the case of video communication in which real-time performance is important, the allowable delay Tth is set to be short.
[0044] In step S2, the adjustment unit 13 acquires the transmission delay between the transmitting device 1 and the receiving device 2. The transmission delay may be a known value, or the transmission delay measured by the measurement unit 3 may be acquired.
[0045] In step S3, the adjustment unit 13 determines a combination of signal processing to be applied to the video signal transmitted from the transmitting device 1 to the receiving device 2 based on the allowable delay Tth set in the processing of step S1 and the allowable delay t of each signal processing obtained in the processing of step S2.
[0046] Fig. 5 is a flowchart showing the detailed processing procedure of step S3 shown in Fig. 4. First, in step S11, the adjustment unit 13 extracts candidate solutions from candidate signal processing. Specifically, when candidate signal processing A, B, and C exist as shown in Figs. 2 and 3, the adjustment unit 13 extracts candidate solutions in which the same type of signal processing (e.g., A1 and A2) do not overlap, and in which the order of the combinations is not the candidate solution. In the example shown in Fig. 3, m candidate solutions are extracted.
[0047] In step S12, the adjustment unit 13 acquires the processing delay t and quality q (see FIG. 2) of each signal processing, and calculates the processing delay T and quality Q of each candidate solution based on these values. In order to reduce the processing time, values measured in advance may be used as the processing delay T and quality Q.
[0048] In step S13, the adjustment unit 13 excludes candidate solutions for which the sum of the processing delay T and the transmission delay exceeds the allowable delay Tth. For example, the combinations "A1-B1-C2" and "A1-B1-C3" shown in FIG. 3 satisfy "T>Tth", so these are excluded.
[0049] In step S14, the adjuster 13 extracts a predetermined number (for example, five) of candidate solutions with the highest quality Q from among the candidate solutions.
[0050] In step S15, the adjustment unit 13 obtains actual measurement values of the processing delay T and the quality Q based on the predetermined number of candidate solutions extracted in the processing of step S14. Specifically, signal processing is applied to the video signal using a combination of signal processing that results in each candidate solution, and an actual measurement value of the processing delay is obtained from the result of this signal processing. An actual measurement value of the quality is also obtained. The adjustment unit 13 selects a candidate solution whose sum of the actual measurement value of the processing delay and the transmission delay is equal to or less than the allowable delay Tth and whose actual measurement value of the quality is the highest, and outputs this candidate solution to the signal processing unit 12.
[0051] The signal processing unit 12 performs signal processing on the video signal based on the combination and order of signal processing that constitutes the candidate solution output from the adjustment unit 13. The processed video signal is then transmitted from the transmission unit 14 and received by the reception unit 21 of the receiving device 2. The video signal received by the reception unit 21 is displayed on the display unit 22.
[0052] Thus, the transmission system 100 of this embodiment is a transmission system 100 comprising a transmitting device 1 that adds any signal processing selected from a plurality of signal processing methods to a video signal and transmits the signal, and a receiving device 2 that receives the transmitted video signal, wherein the transmitting device 1 comprises an adjustment unit 13 that selects one or more combinations of signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay T is less than or equal to a predetermined allowable delay Tth, a signal processing unit 12 that applies the signal processing selected by the adjustment unit 13 to the video signal, and a transmitting unit 14 that transmits the signal-processed video signal, and the receiving device 2 comprises a receiving unit 21 that receives the transmitted video signal and a display unit 22 that displays the received video signal.
[0053] In this embodiment, the combination and order of signal processing applied to the video signal are set so that the sum of the total processing delay T of the combination of signal processing applied to the video signal and the transmission delay is less than or equal to the allowable delay Tth, thereby enabling users to carry out video communication using the transmission system 100 without any discomfort.
[0054] Furthermore, the candidate solution with the highest quality Q is selected from among the candidate solutions for which the sum of the processing delay T and the transmission delay is equal to or less than the allowable delay Tth, thereby making it possible to maintain the high quality of the video signal and shorten the processing delay T. Furthermore, it becomes possible to select appropriate signal processing according to the allowable delay Tth set according to the use case of video communication.
[0055] Second Embodiment Next, a second embodiment will be described. The system configuration is the same as that shown in Fig. 1. In the second embodiment, the adjustment unit 13 selects a combination of signal processing by selecting, from among a plurality of signal processing processes, a plurality of signal processing processes with higher quality within a range in which the sum of the transmission delay and the total processing delay does not exceed the allowable delay.
[0056] The second embodiment differs from the first embodiment in the processing of step S3 shown in Fig. 4. Fig. 6 is a flowchart showing the detailed processing procedure of step S3 shown in Fig. 4. In the second embodiment, an existing optimization method is adopted as a method for determining a combination of signal processing. Here, an example using a greedy method will be described as an example of the optimization method.
[0057] The processing procedure of the transmission system 100 according to the second embodiment will be described below with reference to the flowchart shown in Fig. 6. First, in step S31, the adjustment unit 13 adds the signal processing having the highest quality q from among the candidate signal processing operations to a candidate combination of signal processing operations. For example, it is assumed that, of the signal processing operations A (A1 to A3) shown in Fig. 2, signal processing operation A1 has the highest quality q1. In this case, signal processing operation A1 is added to the candidate combinations.
[0058] In step S32, the adjustment unit 13 determines whether the sum of the total processing delays of the candidate signal processing combinations and the transmission delay exceeds the allowable delay Tth. If the sum of the total processing delays and the transmission delay exceeds the allowable delay Tth (S32; YES), the process proceeds to step S33; if not (S32; NO), the process proceeds to step S34.
[0059] In step S33, the adjustment unit 13 deletes the added signal processing from the combination candidates and execution candidates. That is, if the sum of the processing delay t1 and the transmission delay of the signal processing A1 added in the processing of step S31 exceeds the allowable delay Tth, the adjustment unit 13 deletes this signal processing A1 from the combination candidates and execution candidates.
[0060] In step S34, the adjustment unit 13 deletes signal processing having the same type of technique as the added signal processing from the candidates for execution. That is, if the sum of the processing delay t1 and the transmission delay of the signal processing A1 added in the processing of step S31 is equal to or less than the allowable delay Tth, there is no need to execute signal processing A2 and A3 of the same type as this signal processing A1, and therefore the signal processing A2 and A3 are deleted from the candidates for execution. This prevents the same type of signal processing from being selected more than once.
[0061] In step S35, the adjustment unit 13 determines whether or not there is a candidate signal processing to be executed. If there is (S35; YES), the process returns to step S31.
[0062] For example, if the signal processing A1 is added to the candidate combinations and there is also a candidate signal processing B to be executed, the signal processing B (B1, B2) with the highest quality q (this will be referred to as B1) is added to the candidate combinations, and then steps S32 to S34 are repeated.
[0063] That is, it is determined whether the sum of the processing delay t1 of signal processing A1 and the processing delay t2 of signal processing B1, plus the transmission delay, exceeds the allowable delay Tth. If the sum of the processing delays t1 and t2 and the transmission delay exceeds the allowable delay Tth, signal processing B1 is removed from the candidate combinations in step S33. If the sum of the processing delays t1 and t2 and the transmission delay is equal to or less than the allowable delay Tth, signal processing B2, which is the same type as signal processing B1, is removed from the candidates for execution in step S34.
[0064] On the other hand, if it is determined in the processing of step S35 that there is no signal processing that is a candidate for execution (S35; NO), in step S36 the adjustment unit 13 acquires the actual measured value of the processing delay when processing based on the candidate combination of signal processing is executed and the video signal is transmitted.
[0065] In step S37, the adjustment unit 13 determines whether the sum of the actual measurement value of the processing delay and the transmission delay acquired in step S36 exceeds the allowable delay Tth. If the sum of the actual measurement value of the processing delay and the transmission delay exceeds the allowable delay Tth (S37; YES), the process proceeds to step S38; if not (S37; NO), the process proceeds to step S39.
[0066] In step S38, the adjustment unit 13 deletes the signal processing with the lowest quality q from the combination of signal processing, and then returns to step S36. That is, if the sum of the actual measurement value of the processing delay and the transmission delay exceeds the allowable delay Tth, the adjustment unit 13 deletes the signal processing with the lowest quality so that the sum of the actual measurement value and the transmission delay is equal to or less than the allowable delay Tth.
[0067] In step S39, the adjustment unit 13 determines the current candidate combination of signal processing as the combination of signal processing. In this way, it is possible to set a combination of signal processing with higher quality, without the sum of the total processing delay T and the transmission delay exceeding the allowable delay Tth.
[0068] In this way, in the transmission system according to the second embodiment, as in the first embodiment, the combination and order of signal processing applied to the video signal are set so that the sum of the total processing delay T of the combination of signal processing applied to the video signal and the transmission delay is equal to or less than the allowable delay T. This allows the user to carry out video communication using the transmission system 100 without feeling any discomfort.
[0069] Furthermore, since the signal processing included in the combination is selected in descending order of quality, it is possible to maintain the high quality of the video signal and shorten the processing delay T.
[0070] In the second embodiment, an example in which the greedy method shown in FIG. 6 is used as the optimization method is shown, but an optimization method other than the greedy method may also be adopted.
[0071] [Description of the Third Embodiment] Next, a third embodiment will be described. The device configuration is the same as that shown in FIG. 1 described above. FIG. 7 is a flowchart showing the detailed processing procedure of step S3 shown in FIG. 4. The flowchart shown in FIG. 7 differs from the flowchart shown in FIG. 6 in that steps S51 to S54 have been added. The processing of steps S61 to S69 shown in FIG. 7 is the same as the processing of steps S31 to S39 shown in FIG. 6.
[0072] 7, the adjustment unit 13 sets a first threshold value α and a second threshold value β (β>α). The adjustment unit 13 compares each of the threshold values α and β with the allowable delay Tth. If "Tth<α", the process proceeds to step S52. If "α≦Tth<β", the process proceeds to step S53. If "β≦Tth", the process proceeds to step S54.
[0073] In step S52, the adjustment unit 13 sets the noise removal method as a candidate for execution of signal processing, and then proceeds to step S61.
[0074] In step S53, the adjustment unit 13 sets the noise reduction method and the super-resolution method (a method for processing to increase resolution) as candidates for signal processing, and then proceeds to step S61.
[0075] In step S54, the adjustment unit 13 sets all of the signal processing as candidates for execution of the signal processing. Then, the process proceeds to step S61. Then, the process proceeds to step S61.
[0076] As described above, the processes in steps S61 to S69 are the same as the processes in steps S31 to S39 shown in FIG. 6, and therefore a description thereof will be omitted.
[0077] In the transmission system 100 according to the third embodiment, a first threshold value α and a second threshold value are set for the processing delay T, and if the allowable delay Tth is less than the first threshold value α, the noise reduction method is set as an execution candidate. If the allowable delay Tth is in the range from the first threshold value α to the second threshold value β, the noise reduction method is set as an execution candidate and the super-resolution method. If the allowable delay Tth is equal to or greater than the second threshold value β, all signal processing methods are set as execution candidates.
[0078] Therefore, depending on the magnitude of the allowable delay Tth, the noise reduction method is given priority in signal processing, followed by the super-resolution method, making it possible to give priority to signal processing of higher importance as the signal processing to be applied to the video signal.
[0079] The transmitting device 1 of the present embodiment described above can be, for example, a general-purpose computer system including a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906, as shown in Fig. 8. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of the transmitting device 1.
[0080] The transmission device 1 may be implemented by one computer or by multiple computers, or may be a virtual machine implemented on a computer.
[0081] The program for the transmission device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc), or can be distributed via a network. The computer-readable recording medium is, for example, a non-transitory recording medium.
[0082] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the present disclosure.
[0083] REFERENCE SIGNS LIST 1 transmitter 2 receiver 3 measurement unit 11 acquisition unit 12 signal processing unit 13 adjustment unit 14 transmitter 21 receiver 22 display unit 100 transmission system
Claims
1. A transmitting device that adds any signal processing selected from a plurality of signal processing methods to a video signal and transmits the signal, comprising: an adjustment unit that selects one or a combination of multiple signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay; a signal processing unit that adds the signal processing selected by the adjustment unit to the video signal; and a transmitting unit that transmits the signal-processed video signal.
2. The transmitting device according to claim 1, wherein the adjustment unit selects a combination of signal processing that maximizes the total quality of the combined signal processing.
3. The transmitter of claim 2, wherein the total quality includes a numerical value obtained by adding up the individual qualities of each signal processing included in the selected combination.
4. The transmitting device according to claim 1 or 2, wherein, when there are a plurality of signal processing of the same type, the adjustment unit selects one of the signal processing of the same type.
5. The transmitting device according to claim 1 or 2, wherein the adjustment unit selects, from among a plurality of signal processing methods, a plurality of signal processing methods having higher quality within a range in which the sum of the transmission delay and the total processing delay does not exceed the allowable delay, and selects the combination of signal processing methods.
6. A transmission system comprising a transmitting device which adds any signal processing selected from a plurality of signal processing methods to a video signal and transmits the video signal, and a receiving device which receives the transmitted video signal, wherein the transmitting device comprises: an adjustment unit which selects one or a combination of multiple signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay; a signal processing unit which adds the signal processing selected by the adjustment unit to the video signal; and a transmitting unit which transmits the signal-processed video signal, and the receiving device comprises: a receiving unit which receives the transmitted video signal, and a display unit which displays the received video signal.
7. A transmission method for adding any signal processing selected from a plurality of signal processing methods to a video signal and transmitting the signal, comprising the steps of: selecting one or a combination of multiple signal processing methods from the plurality of signal processing methods so that the sum of the transmission delay and the total processing delay is equal to or less than a predetermined allowable delay; applying the selected signal processing to the video signal; and transmitting the signal-processed video signal.
8. A program for causing a computer to function as the transmitting device according to claim 1 or 2.
Citation Information
Patent Citations
Video distribution system and decoder, and video distribution method
JP2014187448A
Image data transfer device and image data transfer method
JP2021057767A
