Video control program, information processing device, and video control method

Abstract

To provide a video control program capable of suppressing wireless quality deterioration, an information processor, and a video control method.SOLUTION: A video control program includes the steps of: detecting characteristics of the shield that shields the wireless transmission path of video data from video data picked up by an imaging unit; calculating the shielding time of the wireless transmission path by the shield from the detected characteristics; determining control information used for controlling the video data from a first information representing the video quality at the destination of the video data and a second information of the radio quality at the source of the video data based on the calculated shielding time; and controlling the transmission of the video data to the destination using the determined control information.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a video control program, an information processing device, and a video control method. [Background technology]

[0002] In recent years, with the increasing capacity of wireless communication standards such as 5G (fifth generation mobile communications), there has been an increasing demand for wireless transmission of high-definition video such as 4K video and 8K video.

[0003] Specifically, for example, in the field of road monitoring aimed at preventing disasters on roads (e.g., detecting falling objects or vehicles driving in the wrong direction) and in the field of factory monitoring aimed at detecting early signs and improving work efficiency in factories, it is expected that remote monitoring will be realized in which high-definition images captured by an imaging device such as a camera are wirelessly transmitted to a remote server (hereinafter also referred to as a remote server), and the remote server analyzes the images using AI (Artificial Intelligence) (see, for example, Patent Documents 1 to 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-301309 A [Patent Document 2] JP 2010-258850 A [Patent Document 3] JP 2016-105536 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned wireless communication, for example, if a nearby person or object (hereinafter also referred to as an obstruction) gets between the transmitter and the receiver, the quality of the wireless communication may deteriorate and the image displayed on the receiver side may become distorted.

[0006] In this case, video control (hereinafter also referred to as feedback control) is performed between the transmitter and receiver, involving, for example, sending and receiving information about the quality of the video on the receiver side (hereinafter referred to as video quality information), and adjustments are made to the frame rate (fps: frames per second) of the video transmitted from the transmitter to the receiver.

[0007] However, in the above feedback control, it takes time for the receiver to detect the deterioration of the quality of the wireless communication, which may cause a delay in adjusting the frame rate of the video, and may further deteriorate the quality of the wireless communication between the transmitter and the receiver.

[0008] Therefore, for example, on the video transmitter side, video control (hereinafter also referred to as feedforward control) may be performed by using information on wireless quality (hereinafter also referred to as wireless quality information) directly acquired from the transmitter. In this case, since the receiver side does not detect quality deterioration of wireless communication, it is possible to reduce the time required for video control compared to the case where feedback control is performed, and it is possible to suppress quality deterioration of wireless quality.

[0009] In the wireless communication described above, for example, when a transmitter or an obstruction moves, fading (instantaneous fluctuation) occurs. The fluctuation interval of such fading may be shorter than the time required for feedforward control. Therefore, when fading occurs, feedback control may be more effective than feedforward control in suppressing deterioration of wireless quality.

[0010] Therefore, in one aspect, the present invention has an object to provide a video control program, an information processing device, and a video control method that make it possible to suppress deterioration of wireless quality. [Means for solving the problem]

[0011] In one aspect of the embodiment, a feature of an obstruction blocking a wireless transmission path of video data captured by an imaging device is detected from the video data, a blocking time of the wireless transmission path caused by the obstruction is calculated from the detected feature, control information to be used for controlling the video data is determined from first information indicating video quality at a destination of the video data and second information indicating wireless quality at a source of the video data based on the calculated blocking time, and transmission of the video data to the destination is controlled using the determined control information. The processing is executed by a computer. Effect of the Invention

[0012] According to one aspect, it is possible to suppress deterioration of wireless quality. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating the configuration of an information processing system 10. As shown in FIG. [Diagram 2] FIG. 2 is a diagram illustrating the configuration of the information processing system 10. As shown in FIG. [Diagram 3] FIG. 3 is a diagram illustrating the feedback control. [Figure 4] FIG. 4 is a sequence chart illustrating the feedback control. [Diagram 5] FIG. 5 is a diagram illustrating the feedforward control. [Figure 6] FIG. 6 is a sequence chart illustrating the feedforward control. [Figure 7] FIG. 7 is a diagram illustrating the hardware configuration of the information processing device 1. As shown in FIG. [Figure 8] FIG. 8 is a diagram for explaining the hardware configuration of the edge device 2. As shown in FIG. [Figure 9] FIG. 9 is a functional block diagram of the edge device 2. As shown in FIG. [Figure 10] FIG. 10 is a functional block diagram of the information processing device 1. As shown in FIG. [Figure 11] FIG. 11 is a flowchart illustrating an outline of the video control process in the first embodiment. [Figure 12] FIG. 12 is a flowchart illustrating details of the video control process in the first embodiment. [Figure 13] FIG. 13 is a flowchart illustrating details of the video control process in the first embodiment. [Figure 14] FIG. 14 is a flowchart illustrating details of the video control process in the first embodiment. [Figure 15] FIG. 15 is a flowchart illustrating details of the video control process in the first embodiment. [Figure 16] FIG. 16 is a functional block diagram of the edge device 2. As shown in FIG. [Figure 17] FIG. 17 is a functional block diagram of the information processing device 1. [Figure 18] FIG. 18 is a flowchart illustrating the video control process in the second embodiment. [Figure 19] FIG. 19 is a flowchart illustrating the video control process in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] [Configuration of the information processing system in the first embodiment] First, a description will be given of the configuration of the information processing system 10. Figures 1 and 2 are diagrams for explaining the configuration of the information processing system 10.

[0015] 1, the information processing system 10 includes, for example, an information processing device 1 (remote server 1), an edge device 2, an edge device 3, and an edge device 4. In the following description, the information processing system 10 is described as including the edge device 2, the edge device 3, and the edge device 4, but the information processing system 10 may include edge devices other than three.

[0016] The edge devices 2, 3, and 4 are, for example, PCs (Personal Computers) mounted on robots that can move within a factory. The edge device 2 has, for example, a transmitter 2a and a camera 2b, the edge device 3 has, for example, a transmitter 3a and a camera 3b, and the edge device 4 has, for example, a transmitter 4a and a camera 4b. Each of the transmitters 3a, 3a, and 3c is, for example, a communication device equipped with a WiFi chip.

[0017] Then, the edge device 2 captures video data showing, for example, workers and machinery in a factory using the camera 2b, and transmits the captured video data from the transmitter 2a to the information processing device 1 by wireless communication such as WiFi. Similarly, the edge device 3 transmits video data captured by the camera 3b from the transmitter 3a to the information processing device 1, and the edge device 4 transmits video data captured by the camera 4b from the transmitter 4a to the information processing device 1.

[0018] The information processing device 1 is, for example, a physical machine having a receiver 1a, and receives video data transmitted from each of the transmitters 2a, 3a, and 4a. The receiver 1a is, for example, a communication device equipped with a WiFi chip.

[0019] Then, the information processing device 1, for example, inputs the received video data into the trained machine learning model to obtain information indicating whether or not a predetermined abnormality (for example, a worker in poor physical condition or a machine tool with an abnormality in its operation) is captured in the video data. After that, the information processing device 1 outputs information indicating whether or not an abnormality is included in the video data to, for example, an operation terminal (not shown) that is viewed by an administrator.

[0020] Here, as shown in Fig. 2, for example, if an obstruction 5 (such as a person or a container) is between the transmitter 2a and the receiver 1a, the quality of the wireless communication may deteriorate, causing a disturbance in the image displayed on the receiver 1a side. Furthermore, in this case, for example, the quality of the wireless communication may be further deteriorated due to feedback control performed between the transmitter 2a and the receiver 1a. The feedback control will be described below.

[0021] [Feedback control] Fig. 3 is a diagram for explaining the feedback control. Fig. 4 is a sequence chart diagram for explaining the feedback control. The horizontal axis in Fig. 3 indicates time, the vertical axis in the upper part of Fig. 3 indicates RSSI (Received Signal Strength Indicator) indicating the reception strength of a radio signal received by the receiver 1a from the transmitter 2a, the vertical axis in the middle part of Fig. 3 indicates the reception fps in the receiver 1a, and the vertical axis in the lower part of Fig. 3 indicates the transmission fps in the transmitter 2a. The feedback control performed between the transmitter 2a and the receiver 1a will be described below.

[0022] Specifically, when the RSSI decreases as shown between points P11 and P12 in the graph in the upper part of Fig. 3, the reception fps at the receiver 1a decreases as shown between points P21 and P22 in the graph in the middle part of Fig. 3. Then, when the information processing device 1 detects a decrease in the reception fps at the receiver 1a (S1-1 and S1-2 in Fig. 4), it transmits a feedback notification to the edge device 2 (S1-3 in Fig. 4). Specifically, the information processing device 1 transmits, for example, information including the detected reception fps (video quality information) as the feedback notification.

[0023] Thereafter, when the edge device 2 receives a feedback notification from the information processing device 1 (S1-4 in FIG. 4), it performs video control to reduce the transmission fps (S1-5 in FIG. 4) as shown between points P31 and P32 in the graph in the lower part of FIG. 3. Then, when video control to reduce the transmission fps is performed in the transmitter 2a, the reception fps in the receiver 1a partially recovers as shown between points P23 and P24 in the graph in the middle part of FIG.

[0024] That is, the example shown in FIG. 3 indicates that the received fps decreases further between points P22 and P23 in the graph in the middle of FIG. 3, for example, due to the execution of feedback control in response to a decrease in RSSI.

[0025] On the other hand, in the edge device 2, instead of the feedback control, for example, feedforward control using wireless quality information acquired from the transmitter 2a may be performed. In this case, since the receiver 1a does not detect the quality deterioration of the wireless communication, it is possible to reduce the time required for video control compared to the case where feedback control is performed, and it is possible to suppress the quality deterioration of the wireless quality. The feedforward control will be described below.

[0026] [Feedforward control] Fig. 5 is a diagram for explaining the feedforward control. Fig. 6 is a sequence chart for explaining the feedforward control. The feedforward control performed between the transmitter 2a and the receiver 1a will be explained below.

[0027] Specifically, when the RSSI decreases as shown between points P41 and P42 in the graph in the upper part of Fig. 5, the reception fps at the receiver 1a decreases as shown between points P51 and P52 in the graph in the middle part of Fig. 5. On the other hand, when the transmitter 2a detects a decrease in the RSSI (S2-1 in Fig. 6), it spontaneously transmits a feedforward notification to the edge device 2 (S2-2 in Fig. 6). Specifically, the transmitter 2a transmits, for example, information including the RSSI (wireless quality information) as the feedforward notification.

[0028] Thereafter, when the edge device 2 receives a feedforward notification from the transmitter 2a (S2-3 in FIG. 6), it performs video control to reduce the transmission fps (S2-4 in FIG. 6) as shown between points P61 and P62 in the graph in the lower part of FIG. 5. Then, when the video control to reduce the transmission fps is performed in the transmitter 2a, the reception fps in the receiver 1a partially recovers as shown between points P53 and P54 in the graph in the middle part of FIG.

[0029] That is, the examples shown in Figures 5 and 6 show that it is possible to shorten the time during which the receiving fps drops significantly (for example, the time between points P52 and P53 in the graph in the middle of Figure 5) more than in the cases described in Figures 3 and 4.

[0030] In the wireless communication described above, for example, when the transmitter 2a or the shield 5 moves, fading (instantaneous fluctuation) occurs. The fluctuation interval of such fading may be, for example, several hundred milliseconds to several seconds, which may be shorter than the time required for feedforward control. Therefore, when fading occurs, feedback control may be more effective than feedforward control in suppressing deterioration of wireless quality.

[0031] Therefore, the edge device 2 in this embodiment detects the characteristics of the obstruction 5 that obstructs the wireless transmission path of the video data (the wireless transmission path between the transmitter 2a and the receiver 1a) from the video data captured by the camera 2b, for example. Then, the edge device 2 calculates the obstruction time of the wireless transmission path by the obstruction 5 from the detected characteristics.

[0032] Then, based on the calculated occlusion time, the edge device 2 determines control information to be used for controlling the video data from video quality information (hereinafter also referred to as first information) at the destination of the video data (receiver 1a) and wireless quality information (hereinafter also referred to as second information) at the source of the video data (transmitter 2a).Then, the edge device 2 controls the transmission of the video data to the destination using the determined control information.

[0033] That is, the edge device 2 in this embodiment uses different control information for video control of video data depending on the duration of blocking of the wireless transmission path by the blocking object 5 calculated from the characteristics of the blocking object 5.

[0034] This allows the edge device 2 in the present embodiment to perform video control of video data in accordance with the size of the obstruction 5 located on the wireless transmission path, etc. Therefore, the edge device 2 can suppress deterioration of wireless quality.

[0035] [Hardware configuration of information processing system] Next, the hardware configuration of the information processing system 10 will be described. Fig. 7 is a diagram for explaining the hardware configuration of the information processing device 1. Fig. 8 is a diagram for explaining the hardware configuration of the edge device 2. Note that the hardware configurations of the edge devices 3 and 4 are the same as that of the edge device 2, so a description thereof will be omitted.

[0036] First, the hardware configuration of the information processing device 1 will be described.

[0037] 7, the information processing device 1 includes a CPU 101 which is a processor, a memory 102, an I / O interface 103, and a storage medium 104. Each unit is connected to one another via a bus 105.

[0038] The storage medium 104 has, for example, a program storage area (not shown) that stores a program 110 for performing a process for controlling video data (hereinafter also referred to as a video control process). The storage medium 104 also has, for example, an information storage area 130 that stores information used when performing the video control process. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0039] The CPU 101 executes a program 110 loaded from the storage medium 104 into the memory 102 to perform video control processing.

[0040] The I / O interface 103 is, for example, an interface device such as a network interface card, and is capable of accessing the edge device 2 and the like via the receiver 1a.

[0041] Next, the hardware configuration of the edge device 2 will be described.

[0042] 8, the edge device 2 includes a CPU 201 which is a processor, a memory 202, an I / O interface 203, and a storage medium 204. Each unit is connected to one another via a bus 205.

[0043] The storage medium 204 has, for example, a program storage area (not shown) that stores a program 210 for performing the video control processing. The storage medium 204 also has, for example, an information storage area 230 that stores information used when performing the video control processing. The storage medium 204 may be, for example, an HDD or an SSD.

[0044] The CPU 201 executes a program 210 loaded from the storage medium 204 into the memory 202 to perform video control processing.

[0045] The I / O interface 203 is, for example, an interface device such as a network interface card, and is capable of accessing the information processing device 1 and the like via the transmitter 2a.

[0046] [Functions of information processing systems] Next, the functions of the information processing system 10 will be described. Fig. 9 is a functional block diagram of the edge device 2. Fig. 10 is a functional block diagram of the information processing device 1. Note that the functional block diagrams of the edge devices 3 and 4 are the same as the functional block diagram of the edge device 2, so descriptions thereof will be omitted.

[0047] As shown in FIG. 9, the edge device 2 realizes various functions including a video receiving unit 211, a feature detection unit 212, a time calculation unit 213, a control decision unit 214, a video control unit 215, and a video transmission unit 216 by organic cooperation between hardware such as a CPU 201 and a memory 202 and a program 210.

[0048] The video receiving unit 211 receives, for example, video data captured by the camera 2b.

[0049] The feature detection unit 212 determines, for example, from the video data received by the video receiving unit 211, whether or not there is a shield 5 blocking the wireless transmission path between the transmitter 2a and the receiver 1a of the information processing device 1. Then, when the video detection unit 212 determines, for example, that there is a shield 5 blocking the wireless transmission path, it detects the features of the shield 5.

[0050] The time calculation unit 213 calculates the blocking time of the wireless transmission path caused by the blocking object 5 from the characteristics of the blocking object 5 detected by the characteristics detection unit 212, for example.

[0051] The control decision unit 214 decides control information to be used for controlling the video data from first information indicating the video quality of the video data at the receiver 1a and second information indicating the wireless quality at the transmitter 2a, for example, based on the occlusion time calculated by the time calculation unit 213.

[0052] The video control unit 215 performs video control between the transmitter 2a and the receiver 1a, for example, by using the control information determined by the control determination unit 214. This video control includes, in addition to adjusting the frame rate, for example, adjusting the compression rate of the video data transmitted from the transmitter 2a.

[0053] The video transmission unit 216 performs control so that, for example, the video data received by the video reception unit 211 is transmitted by the transmitter 2a.

[0054] In addition, as shown in FIG. 10, the information processing device 1 realizes various functions including a video receiving unit 111, a video analysis unit 115, and a video output unit 116 by organic cooperation between hardware such as a CPU 101 and a memory 102 and a program 110.

[0055] The video receiving unit 111 performs control so that the receiver 1a receives, for example, video data transmitted from the transmitter 2a.

[0056] The video analysis unit 115, for example, inputs the video data received by the receiver 1a into a trained machine learning model, thereby determining whether or not a predetermined abnormality is captured in the video data.

[0057] For example, the video output unit 116 displays the video data received by the receiver 1a on the administrator's operation terminal. Also, for example, the video output unit 116 displays the determination result by the video analysis unit 115 on the administrator's operation terminal.

[0058] [Outline of the first embodiment] Next, an outline of the first embodiment will be described. Fig. 11 is a flow chart for explaining an outline of the video control process in the first embodiment. Below, an outline of the video control process performed in the edge device 2 will be described.

[0059] As shown in FIG. 11, the edge device 2 waits until it receives video data captured by the camera 2b (NO in S11), for example.

[0060] Then, when the edge device 2 receives the video data captured by the camera 2b (YES in S11), the edge device 2 detects, for example, from the video data received in the processing of S11, the characteristics of the obstruction 5 that blocks the wireless transmission path between the transmitter 2a and the receiver 1a of the information processing device 1 (S12).

[0061] Next, the edge device 2 calculates the obstruction time of the wireless transmission path caused by the obstruction 5 based on the characteristics of the obstruction 5 detected in the process of S13 (S13).

[0062] Furthermore, the edge device 2 determines control information to be used for controlling the video data from first information indicating the video quality at the receiver 1a of the video data and second information indicating the wireless quality at the transmitter 2a, for example, based on the occlusion time calculated in the processing of S13 (S14).

[0063] Thereafter, the edge device 2 controls the transmission of video data from the transmitter 2a to the receiver 1a by using, for example, the control information determined in the process of S14 (S15).

[0064] This allows the edge device 2 in the present embodiment to control the video data in accordance with the size of the obstruction 5 located on the wireless transmission path, etc. Therefore, the edge device 2 can efficiently suppress deterioration of wireless quality.

[0065] [Details of the first embodiment] Next, the details of the first embodiment will be described. Fig. 12 to Fig. 15 are flow charts for explaining the details of the video control process in the first embodiment. The details of the video control process performed in the information processing device 1 and the edge device 2 will be described below.

[0066] [Main processing of video control] First, the main processing of the video control processing will be described. Figures 12 and 13 are flow charts illustrating the main processing of the video control processing in the first embodiment.

[0067] As shown in FIG. 12, the video receiving unit 211 of the edge device 2 waits until it receives video data (frames constituting the video data) captured by the camera 2b (NO in S21).

[0068] Then, when the video data captured by the camera 2b is received (YES in S21), the feature detection unit 212 of the edge device 2 detects, for example, from the video data received in the processing of S11, the features of the obstruction 5 that blocks the wireless transmission path between the transmitter 2a and the receiver 1a of the information processing device 1 (S22).

[0069] Specifically, the feature detection unit 212 detects the features of the obstructing object 5 by using a learning model that has learned training data including image data showing each object and features (position and size) of each object. In addition, the feature detection unit 212 detects the features by using, for example, an extraction result of difference pixels when the video data is compressed.

[0070] Next, the feature detection unit 212 determines whether or not the obstruction 5, the feature of which has been detected in the process of S22, is an object that blocks the wireless transmission path for a long period of time (S23).

[0071] Specifically, for example, the feature detection unit 212 determines whether or not the size of the object in which the obstruction 5 was detected in the process of S22 is equal to or larger than a predetermined size.

[0072] As a result, when it is determined that the obstruction 5 whose characteristics are detected in the process of S22 is the obstruction 5 that blocks the wireless transmission path for a long time (YES in S24), the characteristic detection unit 212 determines, for example, whether the obstruction 5 whose characteristics are detected in the process of S22 is located within a first Fresnel zone (S25). The first Fresnel zone is an ellipsoidal region determined from the shortest distance between the transmitter 2a and the receiver 1a.

[0073] Then, when it is determined that the obstruction 5, the characteristics of which were detected in the processing of S22, is located within the first Fresnel zone (YES in S26), the time calculation unit 213 calculates the obstruction time from the passing distance of the obstruction 5, the characteristics of which were detected in the processing of S22, through the first Fresnel zone located between the transmitter 2a and the receiver 1a (hereinafter also simply referred to as the passing distance of the object), the length of the obstruction 5, the characteristics of which were detected in the processing of S22, and the moving speed of the obstruction 5, the characteristics of which were detected in the processing of S22 (S27).

[0074] Specifically, time calculation unit 213 calculates the occlusion time by dividing the sum of the object's passing distance and the object's length by the object's moving speed. More specifically, for example, if the object's passing distance is 1.2 (m), the object's length is 2.0 (m), and the object's moving speed is 2.0 (km / h), time calculation unit 213 calculates 5.8 (s) as the occlusion time.

[0075] On the other hand, if it is determined in the processing of S22 that the obstruction 5 whose characteristics have been detected is not an obstruction 5 that blocks the wireless transmission path for a long period of time (NO in S24), or if it is determined in the processing of S22 that the object whose characteristics have been detected is not located within the first Fresnel zone (NO in S26), the time calculation unit 213 calculates the obstruction time by using the fading frequency in the area between the transmitter 2a and the receiver 1a, as shown in FIG. 13 (S31).

[0076] The fading frequency is, for example, when the transmitter 2a moves, the frequency at which the radio wave strength (the reception strength of the radio signal received by the receiver 1a from the transmitter 2a) changes to such an extent that it affects the image quality on the receiver 1a side due to the relative movement of the transmitter 2a with respect to the receiver 1a and the obstruction 5. In addition, the fading frequency is, for example, when the obstruction 5 moves, the frequency at which the radio wave strength changes to such an extent that it affects the image quality on the receiver 1a side due to the relative movement of the obstruction 5 with respect to the transmitter 2a and the receiver 1a.

[0077] Specifically, the fading frequency is calculated by the following formula (1): d denotes the fading frequency, v denotes the moving speed of a moving object (e.g., the transmitter 2a or the shielding object 5), c denotes the speed of light, and f c indicates the frequency of the radio signal.

[0078] f d =(v / c)f c ...Equation (1) More specifically, for example, when the receiver 1a and the obstruction 5 are not moving, the transmitter 2a is moving at 4 (km / h), and wireless communication is being performed between the transmitter 2a and the receiver 1a using WiFi (5.2 (GHz), W52), the time calculation unit 213 calculates the fading frequency to be 19 (Hz) according to the above equation (1).

[0079] In this case, time calculation section 213 calculates 0.05 (s), which is the reciprocal of the fading frequency.

[0080] Also, for example, when the receiver 1a and the transmitter 2a are not moving, the obstruction 5 is moving at 2 (km / h), and wireless communication is being performed between the transmitter 2a and the receiver 1a using WiFi (5.2 (GHz), W52), the time calculation unit 213 calculates the fading frequency to be 10 (Hz).

[0081] In this case, time calculation section 213 calculates 0.1 (s), which is the reciprocal of the fading frequency.

[0082] Thereafter, the control decision unit 214 determines whether the occlusion time calculated in the process of S27 or S31 is longer than a predetermined threshold (S32). The predetermined threshold may be, for example, a time required for feedback control of the wireless communication performed between the transmitter 2a and the receiver 1a.

[0083] As a result, when it is determined that the occlusion time calculated in the process of S27 or S31 is longer than the predetermined threshold (YES in S33), the control decision unit 214 decides that the first information corresponding to the video quality is the control information (S34).

[0084] On the other hand, if it is determined that the obstruction time calculated in the process of S27 or S31 is not longer than the predetermined threshold (NO in S33), the control decision unit 114 decides that the second information corresponding to the wireless quality is the control information (S35).

[0085] [Video transmission processing] Next, the process of transmitting video data (hereinafter, also referred to as video transmission process) among the video control process will be described. Figures 14 and 15 are flow charts for explaining the video transmission process in the first embodiment.

[0086] First, the video transmission process in the edge device 2 will be described.

[0087] As shown in FIG. 14, the video receiving unit 211 waits until it receives video data captured by the camera 2b (NO in S41).

[0088] Then, when video data captured by camera 2b is received (YES in S41), the video control unit 215 of the edge device 2 determines whether the determined control information (the control information determined in the processing of S34 or S35 performed immediately before) indicates the first information (S42).

[0089] As a result, when it is determined that the control information indicates the first information (YES in S43), the video control unit 215 performs video control using the first information (S44). That is, in this case, the video control unit 215 performs feedback control by using the latest first information transmitted from the information processing device 1. Then, the video transmission unit 216 transmits the video data received in the process of S41 to the information processing device 1 (S46).

[0090] On the other hand, when it is determined that the control information indicates the second information (NO in S43), the video control unit 215 performs video control using the second information (S45). That is, in this case, the video control unit 215 performs feedforward control by using the latest second information transmitted from the transmitter 2a. Then, the video transmission unit 216 transmits the video data received in the process of S41 to the information processing device 1 (S47).

[0091] Next, the video transmission process in the information processing device 1 will be described.

[0092] As shown in FIG. 15, the video receiving unit 111 of the information processing device 1 waits until the receiver 1a receives the video data transmitted from the edge device 2 (NO in S51).

[0093] Then, when the receiver 1a receives the video data (YES in S51), the video analysis unit 115 of the information processing device 1 analyzes the video data received by the video receiving unit 111 in the process of S51, for example.

[0094] Specifically, the video analysis unit 115 determines whether or not a predetermined abnormality is captured in the video data by, for example, inputting the video data received by the receiver 1a to a trained machine learning model.

[0095] Thereafter, the video output unit 116 of the information processing device 1 displays, for example, the analysis result in the process of S52 on the operation terminal of the administrator (S53).

[0096] In this way, the edge device 2 in this embodiment detects the characteristics of the obstruction 5 that obstructs the wireless transmission path of the video data (the wireless transmission path between the transmitter 2a and the receiver 1a) from the video data captured by the camera 2b, for example. Then, the edge device 2 calculates the obstruction time of the wireless transmission path by the obstruction 5 from the detected characteristics.

[0097] Then, the edge device 2 determines control information to be used for controlling the video data from the video quality information (first information) at the video data receiver 1a and the wireless quality information (second information) at the video data transmitter 2a based on the calculated shadowing time. Then, the edge device 2 controls the transmission of the video data to the destination using the determined control information.

[0098] That is, the edge device 2 in this embodiment uses different control information for video control of video data depending on the duration of blocking of the wireless transmission path by the blocking object 5 calculated from the characteristics of the blocking object 5.

[0099] This allows the edge device 2 in the present embodiment to perform video control of video data in accordance with the size of the obstruction 5 located on the wireless transmission path, etc. Therefore, the edge device 2 can suppress deterioration of wireless quality.

[0100] In the above example, the video quality information transmitted from the information processing device 1 (receiver 1a) is information including the reception fps, and the edge device 2 (transmitter 2a) adjusts the transmission fps, but the present invention is not limited to this. Specifically, the wireless quality information may be information including the recognition rate of the learning model used by the video analysis unit 115, and the edge device 2 may adjust the compression rate of the video data.

[0101] Furthermore, the time calculation unit 213 may calculate the occlusion time by using a wireless ray tracing simulator or the like in the process of S31. Specifically, in this case, the time calculation unit 213 may calculate the occlusion time by inputting the positions of the antennas of the transmitter 2a and the receiver 1a and the characteristics of the obstruction 5 (position, size, moving speed, etc.).

[0102] In addition, in the processing of S31, the time calculation unit 213 may calculate the occlusion time by using a learning model that has learned training data including the positions of the antennas of the transmitter 2a and the receiver 1a, the characteristics of the obstruction 5, and the timing of deterioration of the quality of wireless communication.

[0103] [Second embodiment] Next, a second embodiment will be described.

[0104] In the video control process in the second embodiment, unlike the video control process in the first embodiment, the information processing device 1 determines the control information to be used for video control of the video data.

[0105] [Functions of information processing systems] First, the functions of the information processing system 10 will be described. Fig. 16 is a block diagram of the functions of the edge device 2. Fig. 17 is a block diagram of the functions of the information processing device 1. Note that the following describes the points that are different from the first embodiment.

[0106] As shown in FIG. 16, the edge device 2 realizes various functions including a video receiving unit 211, a video control unit 215, and a video transmitting unit 216, by organic cooperation between hardware such as a CPU 201 and a memory 202 and a program 210.

[0107] 17, the information processing device 1 realizes various functions including a video receiving unit 111, a feature detecting unit 112, a time calculating unit 113, a control determining unit 114, a video analyzing unit 115, and a video output unit 116 by organically cooperating with hardware such as a CPU 101 and a memory 102 and a program 110. The feature detecting unit 112, the time calculating unit 113, and the control determining unit 114 have the same functions as the feature detecting unit 212, the time calculating unit 213, and the control determining unit 214 in the first embodiment.

[0108] That is, in the second embodiment, unlike the first embodiment, the feature detection unit 112, the time calculation unit 113, and the control determination unit 114 are realized in the information processing device 1.

[0109] [Main processing of video control] Next, a flowchart illustrating the video control process in the second embodiment will be described. Figures 18 and 19 are flowcharts illustrating the video control process in the second embodiment.

[0110] [Main processing of video control] 18, for example, the video receiving unit 111 of the information processing device 1 waits until the receiver 1a receives video data transmitted from the edge device 2 (NO in S111). That is, the video receiving unit 111 waits until it receives video data captured by the camera 2b of the edge device 2.

[0111] Then, when the receiver 1a receives the video data (YES in S21), the feature detection unit 112 of the information processing device 1 detects, for example, from the video data received in the processing of S111, the features of the obstruction 5 that blocks the wireless transmission path between the transmitter 2a of the edge device 2 and the receiver 1a (S112).

[0112] Next, the feature detection unit 112 determines whether or not the obstruction 5 whose feature has been detected in the process of S112 is an obstruction 5 that blocks the wireless transmission path for a long period of time (S113).

[0113] Specifically, the feature detection unit 212 determines whether or not the size of the obstructing object 5, the feature of which has been detected in the process of S112, is equal to or larger than a predetermined size.

[0114] As a result, when it is determined that the obstruction 5 whose characteristics were detected in the processing of S112 is an obstruction 5 that blocks the wireless transmission path for a long period of time (YES in S114), the characteristic detection unit 112 determines, for example, whether or not the obstruction 5 whose characteristics were detected in the processing of S112 is located within the first Fresnel zone (S115).

[0115] Then, when it is determined that the obstruction 5, whose characteristics were detected in the processing of S112, is located within the first Fresnel zone (YES in S116), the time calculation unit 113 calculates the obstruction time from the passing distance of the obstruction 5, whose characteristics were detected in the processing of S122, through the first Fresnel zone located between the transmitter 2a and the receiver 1a, the length of the obstruction 5, whose characteristics were detected in the processing of S112, and the moving speed of the obstruction 5, whose characteristics were detected in the processing of S112 (S117).

[0116] On the other hand, if it is determined that the obstruction 5 whose characteristics were detected in the processing of S112 is not located within the first Fresnel zone (NO in S116), the time calculation unit 213 calculates the obstruction time by using the fading frequency between the transmitter 2a and the receiver 1a, as shown in FIG. 19 (S121).

[0117] Thereafter, the control decision unit 214 determines whether the occlusion time calculated in the process of S117 or S121 is longer than a predetermined threshold value (S122).

[0118] As a result, when it is determined that the occlusion time calculated in the process of S117 or S121 is longer than the predetermined threshold (YES in S123), the control determination unit 214 determines the first information corresponding to the video quality as the control information (S124).

[0119] On the other hand, if it is determined that the obstruction time calculated in the process of S17 or S121 is not longer than the predetermined threshold (NO in S123), the control decision unit 114 sets the second information corresponding to the wireless quality as the control information (S125).

[0120] Then, for example, after the process of S124 or S125, the control decision unit 114 transmits to the edge device 2 the control information decided in the process of S124 or S125.

[0121] Thereafter, the edge device 2 performs a video transmission process by using the control information transmitted from the information processing device 1.

[0122] The above embodiment can be summarized as follows:

[0123] (Appendix 1) Detecting features of an obstacle blocking a wireless transmission path of the video data captured by an imaging device; Calculating a blocking time of the wireless transmission path caused by the blocking object from the detected characteristics; determining control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated occlusion time; using the determined control information to control transmission of the video data to the destination. A video control program that causes a computer to execute a process.

[0124] (Appendix 2) In Appendix 1: the first information is information acquired from a destination of the video data; A video control program comprising:

[0125] (Appendix 3) In Appendix 1: The second information is information acquired from a transmitter at a transmission source of the video data. A video control program comprising:

[0126] (Appendix 4) In Appendix 1: In the process of determining, when the occlusion time is longer than a predetermined threshold, the first information is determined as the control information, and when the occlusion time is shorter than the predetermined threshold, the second information is determined as the control information. A video control program comprising:

[0127] (Appendix 5) In Appendix 4: the predetermined threshold value is a time required for feedback control of wireless communication between a transmission source and a transmission destination of the video data; A video control program comprising:

[0128] (Appendix 6) In Appendix 1: In the calculation process, when the obstruction has a size equal to or larger than a predetermined size and is located in a first Fresnel zone, a time taken for the obstruction to pass through the first Fresnel zone is calculated as the obstruction time. A video control program comprising:

[0129] (Appendix 7) In Appendix 1: In the calculation process, when the size of the obstruction is less than a predetermined value or when the obstruction is not located in a first Fresnel zone, an inverse of a fading frequency in the wireless transmission path is calculated as the obstruction time. A video control program comprising:

[0130] (Appendix 8) a feature detection unit that detects features of an obstruction that obstructs a wireless transmission path of video data captured by an imaging device; a time calculation unit that calculates a blocking time of the wireless transmission path caused by the blocking object based on the detected feature; a control determination unit that determines control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated occlusion time; a video control unit that controls transmission of the video data to the destination using the determined control information. 23. An information processing apparatus comprising:

[0131] (Appendix 9) In Appendix 8: The control determination unit determines the first information as the control information when the occlusion time is longer than a predetermined threshold, and determines the second information as the control information when the occlusion time is shorter than the predetermined threshold. 23. An information processing apparatus comprising:

[0132] (Appendix 10) Detecting features of an obstacle blocking a wireless transmission path of the video data captured by an imaging device; Calculating a blocking time of the wireless transmission path caused by the blocking object from the detected characteristics; determining control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated occlusion time; using the determined control information to control transmission of the video data to the destination. A video control method comprising causing a computer to execute processing.

[0133] (Appendix 11) In Appendix 10: In the process of determining, when the occlusion time is longer than a predetermined threshold, the first information is determined as the control information, and when the occlusion time is shorter than the predetermined threshold, the second information is determined as the control information. A video control method comprising: [Explanation of symbols]

[0134] 1: Information processing device 2: Edge device 2a: Transmitter 2b: Camera 3: Edge device 3a: Transmitter 3b: Camera 4: Edge device 4a: Transmitter 4b: Camera 5: Covering 10: Information processing system

Claims

1. A method for detecting a feature of an obstruction from image data captured under a situation in which at least one of an imaging device capturing image data and an obstruction blocking a wireless transmission path of the image data is moving, Calculating a blocking time of the wireless transmission path caused by the blocking object from the detected characteristics; determining control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated shadowing time; using the determined control information to control transmission of the video data to the destination. A video control program that causes a computer to execute a process.

2. In claim 1, the first information is information acquired from a destination of the video data; A video control program comprising:

3. In claim 1, The second information is information acquired from a transmitter at a transmission source of the video data. A video control program comprising:

4. In claim 1, In the process of determining, when the occlusion time is longer than a predetermined threshold, the first information is determined as the control information, and when the occlusion time is shorter than the predetermined threshold, the second information is determined as the control information. A video control program comprising:

5. In claim 4, the predetermined threshold value is a time required for feedback control of wireless communication between a transmission source and a transmission destination of the video data; A video control program comprising:

6. In claim 1, In the calculation process, when the obstruction has a size equal to or larger than a predetermined size and is located in a first Fresnel zone, a time taken for the obstruction to pass through the first Fresnel zone is calculated as the obstruction time. A video control program comprising:

7. In claim 1, In the calculation process, when the size of the obstruction is less than a predetermined value or when the obstruction is not located in a first Fresnel zone, an inverse of a fading frequency in the wireless transmission path is calculated as the obstruction time. A video control program comprising:

8. A feature detection unit that detects features of an obstruction from video data captured under a situation in which at least one of an imaging device that captures video data and an obstruction that blocks a wireless transmission path of the video data is moving; a time calculation unit that calculates a blocking time of the wireless transmission path caused by the blocking object based on the detected feature; a control determination unit that determines control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated occlusion time; a video control unit that controls transmission of the video data to the destination using the determined control information.

23. An information processing apparatus comprising:

9. A method for detecting a characteristic of an obstruction from image data captured under a situation in which at least one of an imaging device capturing image data and an obstruction blocking a wireless transmission path of the image data is moving, Calculating a blocking time of the wireless transmission path caused by the blocking object from the detected characteristics; determining control information to be used for controlling the video data from first information indicating a video quality at a destination of the video data and second information indicating a wireless quality at a source of the video data based on the calculated shadowing time; using the determined control information to control transmission of the video data to the destination. A video control method comprising causing a computer to execute processing.

Images