Network processing device, video transmission method, and program

Abstract

The user terminal transmits video of the work based on transmission control information according to priority. [Solution] The network processing device of the embodiment is connected to a user terminal used by a worker performing work at a predetermined work site via a communication network capable of allocating network resource bandwidth. It detects network information including the bandwidth and utilization rate of the communication network, and performs network control including bandwidth control and bandwidth guarantee on the network based on the priority of each task, management information in a work information database that manages the tasks being performed by the worker using the user terminal, and the network information. Based on the content of the network control, it notifies the user terminal of transmission control information including at least one of the video compression ratio or transmission rate according to the priority of the task, receives the video transmitted from the user terminal based on the transmission control information, and stores it in the video storage unit.

Description

Technical Field

[0001] Embodiments of the present invention relate to a network processing device, a video transmission method, and a program.

Background Art

[0002] In recent years, with the expansion of telework and remote work, the need to remotely perform tasks that were conventionally carried out on-site has been increasing. For example, conventionally, veteran supervisors have monitored and supported the work of workers at the work site. If the veteran supervisor could support the work of the workers from a remote site without visiting the work site, labor and manpower savings in the work could be expected.

[0003] As one of the supports for on-site work, for example, when a new worker or the like performs manufacturing work, if a veteran supervisor can determine, notify, and instruct whether the work procedure is being correctly performed even without being present on the spot, it becomes possible to suppress a decrease in yield due to work errors.

[0004] As a method for determining whether a work procedure is correct, a method of comparing with past information (correct data) using a function of detecting human movement such as human skeleton determination or hand tracking is generally used. For example, a video of the work of a veteran worker performed in the past is recorded as the correct work procedure (correct data), and a comparison is made with a video of the current work of the worker, and it becomes possible to determine whether the work procedure is correct based on whether the two are close.

[0005] In skeleton determination and hand tracking for performing such determination, for example, a video of on-site work is taken and converted into data (for example, converted into three-dimensional coordinate data of each skeleton point of the worker in the video) and compared (matched) with the correct data. When trying to implement hand tracking or the like on a field terminal, a high-performance device such as an XR (Extended Reality) goggle that has a hand tracking function as a default is required, and a high processing capacity is required for the field terminal.

[0006] On the other hand, when comparing data with past work data (correct data), especially in the case of video data, it is difficult to perform work analysis on the field terminal side because it requires comparison with a vast amount of data. Therefore, it is necessary to transmit the video of the field work to a server, and then compare the video of the current work status with the past work information database on the server side to confirm the correctness of the work procedure.

[0007] Furthermore, considering the need for immediate feedback to on-site work, it is necessary to provide on-site work video remotely in real time and to remotely determine whether the work procedures are correct. While work procedure analysis may sometimes be performed manually by experienced workers based on on-site video, when there are many on-site workers, it is necessary to use video analysis technologies such as AI (Artificial Intelligence) to perform automated analysis using a system. When configuring such a system for analyzing work procedures remotely, it is ideal to transmit video data from on-site terminals to the server quickly and reliably. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2022-111778 [Patent Document 2] Japanese Patent Publication No. 2016-187176 [Overview of the project] [Problems that the invention aims to solve]

[0009] Furthermore, simply transmitting work video from field terminals to a remote server and storing the video on the server is highly valuable. However, if there are many field terminals, bandwidth congestion due to video transmission can occur, potentially resulting in the inability to transmit some of the video. Also, work usually has a priority order, and it is especially important to transmit the video of high-priority work quickly and reliably.

[0010] Therefore, an embodiment of the present invention aims to provide a network processing device, a video transmission method, and a program that enable the transmission of video of work from a user terminal based on transmission control information according to priority. [Means for solving the problem]

[0011] The network processing device of the embodiment is a network processing device connected to a user terminal used by a worker performing work at a predetermined work site via a communication network capable of allocating bandwidth of network resources, and comprises: a network monitoring unit that detects network information including the bandwidth and utilization rate of the communication network; a network control unit that performs network control including bandwidth control and bandwidth guarantee to the communication network based on the priority of each task, management information in a work information database that manages the tasks performed by the worker using the user terminal, and the network information; a transmission control instruction unit that, based on the content of the network control, notifies the user terminal that transmits video footage of the work at the work site of transmission control information including at least one of the video compression ratio or transmission rate according to the priority of the task; and a video receiving unit that receives video transmitted from the user terminal based on the transmission control information and stores the received video in a video storage unit. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is an overall configuration diagram of the video transmission system according to the first embodiment. [Figure 2] Figure 2 is an explanatory diagram of the first processing sequence in the video transmission system of the first embodiment. [Figure 3] Figure 3 is an explanatory diagram of the second processing sequence in the video transmission system of the first embodiment. [Figure 4] Figure 4 is an overall configuration diagram of the video transmission system according to the second embodiment. [Figure 5] Figure 5 is an explanatory diagram of the first processing sequence in the video transmission system of the second embodiment. [Figure 6] Figure 6 is an explanatory diagram of a second processing sequence in the video transmission system of the second embodiment. [Figure 7] Figure 7 is an explanatory diagram showing an example of the priority of each task for each worker. [Figure 8] Figure 8 is an explanatory diagram showing the state of the change over time of the network bandwidth when the processing of the prior art is executed in the example of Figure 7. [Figure 9] Figure 9 is an explanatory diagram showing the state of the change over time of the network bandwidth when the processing of the second embodiment is executed in the example of Figure 7. [Figure 10] Figure 10 is an explanatory diagram of information regarding bandwidth allocation in the second embodiment. [Figure 11] Figure 11 is a flowchart showing the processing of the server in the second embodiment. [Figure 12] Figure 12 is an overall configuration diagram of the video transmission system of the third embodiment. [Figure 13] Figure 13 is an explanatory diagram of the relationship between the overhead camera and the work area in the third embodiment. [Figure 14] Figure 14 is a diagram showing a configuration example of a conventional video transmission system.

MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, embodiments (first embodiment to third embodiment) of the network processing apparatus, video transmission method, and program of the present invention will be described with reference to the accompanying drawings.

[0014] (Prior Art) To facilitate understanding of the embodiments, the prior art will be described again. Figure 14 is a diagram showing a configuration example of a conventional video transmission system. The user terminal is a device used by a worker who performs work at a predetermined work site.

[0015] The user terminal includes a camera unit, a display unit, a video transmission unit, and a determination result reception unit.

[0016] The camera unit captures images of the target area (work site) and outputs video (video data). The display unit displays various information such as video and text.

[0017] The video transmission unit transmits video to the server. The judgment result receiving unit receives the correctness judgment result of the work from the server and displays it on the display unit.

[0018] The server comprises a video receiving unit, a video analysis unit, a video storage unit, a correct / incorrect determination unit, and a determination result notification unit.

[0019] The video receiving unit receives video transmitted from the user terminal and stores it in the video storage unit. The video analysis unit analyzes the video using predetermined image processing and outputs analysis data.

[0020] The correct / incorrect judgment unit compares the correct data, which is a video of a sample operation, with the analysis data to determine whether the operation is correct or incorrect, and outputs the correct / incorrect judgment result. The judgment result notification unit notifies the user terminal of the correct / incorrect judgment result.

[0021] However, with this simple method of transmitting work video from user terminals to the server, if there are many user terminals, bandwidth congestion due to video transmission can occur, potentially resulting in some video not being transmitted. Furthermore, tasks usually have priorities, and it is especially important to transmit the video of high-priority tasks quickly and reliably.

[0022] Therefore, the following describes a technology that enables the transmission of video footage from a user terminal based on transmission control information according to priority.

[0023] (First Embodiment) Figure 1 is an overall configuration diagram of the video transmission system S of the first embodiment. The video transmission system S comprises user terminals 1 (1a, 1b, 1c) used by workers performing work at a predetermined work site, and a server 2 (network processing device) connected to the user terminals 1 via a network A (communication network) capable of low-latency bandwidth allocation of network resources. Network A may be, for example, a low-latency wired communication network, or a low-latency wireless communication network such as 5G or local 5G.

[0024] User terminal 1 is, for example, smart glasses (glasses-type wearable terminal worn on the head) worn by a worker. User terminal 1 comprises a camera unit 11, a video transmission unit 12, and a transmission control unit 13.

[0025] Furthermore, the user terminal 1 may be equipped with a hardware configuration such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc., to perform various processes. The user terminal 1 may also be equipped with an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), etc.

[0026] The camera unit 11 captures the target area (work site) and outputs video (video data). The camera unit 11 can be any camera capable of capturing images that show the work situation and the movements of the hands. For example, the camera unit 11 could be the camera of smart glasses that captures what is in the line of sight of the worker wearing the smart glasses. Alternatively, the camera unit 11 could be a fixed camera that captures the work situation.

[0027] The video transmission unit 12 transmits video to the server 2. The transmission control unit 13 controls the transmission of video according to instructions from the server 2.

[0028] In addition to the above configuration, user terminal 1 is equipped with various sensors such as acceleration sensors and angular velocity sensors, as well as communication interfaces for communicating with external devices such as server 2, as needed.

[0029] Furthermore, user terminal 1 is not limited to smart glasses; it may also be a smartphone, tablet, or the like.

[0030] Server 2 is a computer and comprises a storage unit 21, a processing unit 22, an input unit 23, and an output unit 24.

[0031] The memory unit 21 is composed of, for example, RAM (Random Access Memory), ROM (Read Only Memory), SSD (Solid State Drive), HDD (Hard Disk Drive), etc., and stores various types of information. The memory unit 21 includes a work information database 211 and an image storage unit 212.

[0032] The work information database 211 is a database that stores management information, including the priority of each task and information that manages the tasks being performed by the worker using the user terminal 1. The video storage unit 212 stores video transmitted from multiple user terminals 1 in a way that allows each user terminal 1 to be identified. The video storage unit 212 also stores correct data (data of the correct work video) for each task.

[0033] The input unit 23 is a means for the user to input information, such as a keyboard, mouse, or touch panel.

[0034] The output unit 24 is a means for outputting information, and is, for example, a display device (e.g., an LCD (Liquid Crystal Display)) or an audio output device (e.g., a speaker).

[0035] The processing unit 22 corresponds to a so-called processor and is composed of, for example, a CPU, MPU, GPU, etc., and performs various processes. Alternatively, the processing unit 22 may be composed of, for example, an integrated circuit such as an ASIC or FPGA.

[0036] The processing unit 22 includes a network monitoring unit 221, a network control unit 222, a transmission control instruction unit 223, and a video receiving unit 224.

[0037] The network monitoring unit 221 detects network information, including the bandwidth and utilization rate of network A.

[0038] The network control unit 222 performs network control, including bandwidth control and bandwidth guarantee, on network A based on management information in the work information database 211 and network information detected by the network monitoring unit 221.

[0039] For example, if the network control unit 222 determines, based on network information, that network A is experiencing bandwidth shortages, it allocates network bandwidth according to the priority of the tasks corresponding to the video.

[0040] Furthermore, the network control unit 222 recognizes the tasks and their priorities corresponding to user terminal 1 based on management information in the work information database 211. If the priority is high, it performs network control for high-quality real-time video, including securing a fixed bandwidth or guaranteeing latency. If the priority is low, the network control unit 222 performs network control with best-effort bandwidth allocation.

[0041] The transmission control instruction unit 223 notifies each user terminal 1 of transmission control information, including at least one of the video compression ratio or transmission rate, according to the priority of the task, based on the network control content (bandwidth control, bandwidth guarantee, etc.) performed by the network control unit 222.

[0042] For example, if the transmission control instruction unit 223 has a high priority, it notifies the user terminal 1 to improve the video quality, and at that time, in conjunction with the network control unit 222's network control for high-quality real-time video, it performs video high-definition control, which includes improving the video resolution, improving the transmission rate, or decreasing the video compression ratio, among other things.

[0043] Furthermore, for example, if the transmission control instruction unit 223 has a high priority, when performing transmission control to the user terminal 1, it performs transmission control in conjunction with the network control unit 222's network control for high-quality real-time video, which includes at least one of the following: a reduction in resolution, a reduction in transmission rate, or an improvement in video compression ratio, within the range that ensures the necessary resolution of the video.

[0044] Furthermore, for example, when the transmission control instruction unit 223 has a low priority, when it performs transmission control to the user terminal 1, it performs transmission control that includes at least one of the following: a decrease in video resolution, a decrease in transmission rate, or an increase in video compression ratio, in conjunction with the network control unit 222's best-effort bandwidth allocation network control.

[0045] The video receiving unit 224 receives video transmitted from the user terminal 1 based on transmission control information and stores the received video in the video storage unit 212.

[0046] Note that while Figure 1 shows a case with three user terminals 1, it is not limited to this; there may be two or four or more user terminals 1.

[0047] Figure 2 is an explanatory diagram of the first processing sequence in the video transmission system S of the first embodiment. Figure 2 shows an example of a communication path in video transmission.

[0048] Each user terminal 1 captures video of the on-site work situation with its camera unit 11 and transmits the video from its video transmission unit 12 to its video receiving unit 224 on server 2 via network A. Server 2 stores the video received by the video receiving unit 224 in its video storage unit 212. Furthermore, when analyzing work procedures, this can be done in real time or later by referring to the work information database 211. The analysis of work procedures can also be performed automatically by AI or other means, or manually.

[0049] Figure 3 is an explanatory diagram of the second processing sequence in the video transmission system of the first embodiment. Figure 3 shows an example of a communication path associated with bandwidth control of network A.

[0050] The network monitoring unit 221 of server 2 periodically monitors the bandwidth usage of network A (S11), and if the usage is high and there is a possibility of bandwidth congestion, it instructs the network control unit 222 to control the bandwidth (S12).

[0051] In response, the network control unit 222 obtains the priority of the current task for each user terminal 1 from the work information database 211 (S13), and performs network bandwidth control on network A according to the priority of the tasks (S14).

[0052] At the same time, the network control unit 222 notifies the transmission control instruction unit 223 to control the transmission quality of each user terminal 1 (S15). The transmission control instruction unit 223 notifies the transmission control unit 13 of each user terminal 1 of the priority information via network A (S16). The transmission control unit 13 of each user terminal 1 changes parameters such as the transmission rate and video compression ratio based on the given priority information.

[0053] Thus, according to the server 2 of the first embodiment, video of the work can be transmitted from the user terminal 1 based on transmission control information according to priority. For example, the server 2 can transmit video of high-priority work from the user terminal 1 quickly and reliably.

[0054] (Second Embodiment) Next, a second embodiment will be described. Explanations of matters similar to those in the first embodiment will be omitted as appropriate. Figure 4 is an overall configuration diagram of the video transmission system S of the second embodiment.

[0055] In addition to the configuration shown in Figure 1, the user terminal 1 further includes a display unit 14 and a judgment result receiving unit 15.

[0056] The display unit 14 displays various types of information, such as images and text.

[0057] The judgment result receiving unit 15 receives the correctness judgment result of the work received from the server 2 and displays the correctness judgment result on the display unit 14.

[0058] Furthermore, in addition to the configuration shown in Figure 1, Server 2 also includes a video analysis unit 225, a correct / incorrect determination unit 226, and a determination result notification unit 227.

[0059] The video analysis unit 225 analyzes the video using predetermined image processing and outputs analysis data. For example, the video analysis unit 225 analyzes the video stored in the video storage unit 212 using image processing and outputs analysis data. Alternatively, the video analysis unit 225 may analyze the video acquired from the video receiving unit 224 using image processing and output analysis data.

[0060] Furthermore, the video analysis unit 225 analyzes the video using image processing and outputs analysis data. From the analysis data, it identifies the task and updates the work information database 211 with information about the task being performed by the worker using the user terminal 1.

[0061] The correct / incorrect determination unit 226 compares the correct data, which is a video of a sample operation, with the analysis data to determine whether the operation is correct or incorrect and outputs the correct / incorrect determination result. For example, the correct / incorrect determination unit 226 outputs a correct / incorrect determination result (e.g., OK / NG) that includes whether or not there is an error in the operation procedure.

[0062] The judgment result notification unit 227 notifies the user terminal 1 of the correct / incorrect judgment result. For example, the judgment result notification unit 227 notifies the user terminal 1 of the correct / incorrect judgment result as information for display. In response, the user terminal 1 displays the correct / incorrect judgment result on the display unit 14.

[0063] Figure 5 is an explanatory diagram of the first processing sequence in the video transmission system of the second embodiment. Figure 5 shows an example of a communication path in video transmission.

[0064] Each user terminal 1 captures video of the work situation at the site using the camera unit 11 and transmits the video from the video transmission unit 12 to the video receiving unit 224 of server 2 via network A (S31). Server 2 sends the video received by the video receiving unit 224 to the video analysis unit (S32), where the video analysis unit 225 performs image processing and saves the video data to the video storage unit 212 (S33).

[0065] The video analysis unit 225 identifies the work process corresponding to the user terminal 1 from the analyzed video and registers it in the work information database 211 (S34). The video analysis unit 225 also instructs the correctness determination unit 226 to make a correctness determination based on the identified work process (S35). The correctness determination unit 226 obtains the work video data of the user terminal 1 from the video storage unit 212 (S36, S37) and video data of previously recorded normal work procedures (correct answer data) that have been stored in advance, and compares them to determine whether the work of the user terminal 1 is correct or not. In addition to video comparison using image processing, the correctness determination of the work (correctness determination) may also be made by quantifying the video using hand tracking or human skeleton determination (for example, converting it into three-dimensional coordinate data of each skeletal point of the worker in the video) and making a correctness determination by comparing the data, and the method of correctness determination is not limited.

[0066] The correct / incorrect determination unit 226 transmits the correct / incorrect determination result to the determination result notification unit 227 (S38). In response, the determination result notification unit 227 notifies the correct / incorrect determination result to the determination result receiving unit 15 of each user terminal 1. The user terminal 1 displays the received correct / incorrect determination result on the display unit 14 to show the user the correct / incorrect determination result of the operation.

[0067] Figure 6 is an explanatory diagram of the second processing sequence in the video transmission system of the second embodiment. Figure 6 shows an example of a communication path associated with bandwidth control of network A.

[0068] The network monitoring unit 221 of server 2 periodically monitors the bandwidth usage of network A (S51), and if the usage is high and there is a possibility of bandwidth congestion, it instructs the network control unit 222 to control the bandwidth (S52).

[0069] The network control unit 222 obtains the current task priority for each user terminal 1 from the work information database 211 (S53), and performs network bandwidth control on network A according to the task priority (S54). At the same time, the network control unit 222 notifies the transmission control instruction unit 223 to control the transmission quality to each user terminal 1 (S55).

[0070] The transmission control instruction unit 223 notifies the transmission control unit 13 of each user terminal 1 of priority information via network A (S56). The transmission control unit 13 of the user terminal 1 changes parameters such as the transmission rate and video compression ratio based on the given priority information.

[0071] Figure 7 is an explanatory diagram illustrating an example of the priority of each task for each worker. Figure 7 shows an example of a task (work process) and its priority.

[0072] Worker α performs tasks A, B, and C. Worker β performs tasks D, E, and F. Worker γ performs tasks G, H, and I. Tasks A, D, and G are high-priority tasks in video analysis, such as tasks of high importance or tasks requiring detailed movements.

[0073] Tasks B, E, and H are low-priority tasks that require relatively little video analysis, where a reduction in video quality is not a problem, and where the occurrence of video distortion or block noise has little impact. Tasks C, F, and I are assumed to be of intermediate priority.

[0074] Furthermore, the start times for each worker (α-γ) may differ. Therefore, it is common for work scenarios to occur where users performing high-priority tasks and users performing low-priority tasks are mixed together in a workflow where tasks are being carried out concurrently.

[0075] Figure 8 is an explanatory diagram showing how network bandwidth changes over time when conventional processing is performed using the example in Figure 7.

[0076] If no controls are applied to network bandwidth or transmission quality, the transmission rate of video captured for each work process will be uniform. For example, let's assume that each user terminal 1 (workers α to γ) always transmits high-definition work video to server 2 using a communication bandwidth of 20 Mbps. Also, let's assume that the upper limit of network A's communication bandwidth is 50 Mbps.

[0077] In this case, tasks A, B, and C, which are performed by worker α, will always occupy a bandwidth of 20 Mbps regardless of priority. Similarly, tasks D, E, and F, which are performed by worker β, will also occupy 20 Mbps. Up to worker β, the bandwidth will remain within the bandwidth limit of network A.

[0078] However, once worker γ starts work G, the sum of the transmission rates of workers α through γ will exceed the network bandwidth limit, resulting in video distortion or loss. Thus, without controlling network bandwidth and transmission quality, unreasonable results can occur, such as the video of the low-priority work B reaching server 2 while the video of the high-priority work G is lost.

[0079] On the other hand, Figure 9 is an explanatory diagram showing the change in network bandwidth over time when the processing of the second embodiment is performed in the example of Figure 7. In other words, Figure 9 shows an example of bandwidth utilization when bandwidth control is applied to network A when transmitting video of the work status of each work process in Figure 7.

[0080] This method identifies the priority of tasks, applies bandwidth control to network A, and controls the video transmission quality of each user terminal 1. For example, for high-priority tasks, a communication bandwidth of 20 Mbps is guaranteed, and the transmission control unit 13 of user terminal 1 is instructed to maintain high-definition video quality.

[0081] Similarly, a 10Mbps bandwidth will be allocated to tasks with intermediate priority, and a 5Mbps bandwidth to tasks with lower priority. Within the allocated bandwidth, video quality, such as video compression ratio, will be defined and allocated accordingly.

[0082] In this case, user terminal 1a (worker α) is allocated a communication bandwidth of 20 Mbps while performing high-priority task A, and transmits high-definition video data for video analysis to server 2. Server 2 analyzes worker α's work video, and when it determines from the analysis of the work procedure that task A has been completed and worker α has moved on to task B, the network control unit 222 of server 2 controls the bandwidth to limit worker α's communication bandwidth to 5 Mbps, as task B has a lower priority, and switches to a low-quality video transmission mode in which the video is transmitted using the limited bandwidth, such as by reducing the video quality of worker α.

[0083] As a result, even when user terminal 1b (worker β) starts task D and user terminal 1c (worker γ) starts task G, worker α's communication bandwidth is limited, making it possible to transmit all work video to server 2 without the bandwidth exceeding the network bandwidth limit.

[0084] Similarly, when worker α completes task B and moves on to task C, a medium-priority communication bandwidth of 10 Mbps is allocated, and the video compression rate is relaxed accordingly to improve the video quality to an intermediate level before transmitting the video to server 2.

[0085] Once task C is completed, the allocation of communication bandwidth to worker α is finished. Similarly, for workers β and γ, the start of work and transition to the next task are determined from video analysis, and communication bandwidth allocation and video quality changes are controlled according to the priority of the work. This adjusts the overall communication bandwidth usage, achieving video transmission that balances video quality assurance with prevention of communication loss.

[0086] Furthermore, the transition between work processes may be automatically detected by video analysis on server 2. Alternatively, workers α to γ ​​can voluntarily operate the work process transition on user terminal 1 and notify server 2 of the start and end of the work from user terminal 1. In other words, the method of detecting the transition between work processes and recognizing the work is not restricted.

[0087] Figure 10 is an explanatory diagram of information regarding bandwidth allocation in the second embodiment. Figure 10 shows an example of a network control method that controls video quality and allocates network bandwidth for each user terminal 1 based on the priority of the work process.

[0088] The network control unit 222 of server 2 determines network priority according to the priority of the task and allocates network resources that enable stable communication. For example, in systems such as 5G communication and local 5G communication, there is a technology called network slicing that divides the network into virtual network resources and allocates them for communication, and such technology is used to guarantee the bandwidth for video data transmission of user terminal 1 that transmits high-definition video data.

[0089] In the example shown in Figure 10, the system instructs user terminal 1c, used by worker γ performing high-priority task G, to transmit uncompressed, high-definition video over a fixed bandwidth, while also allocating broadband network resources (20 Mbps) to guarantee communication quality.

[0090] For user terminal 1a used by worker α, who is performing task C with a medium priority, moderate video compression (low compression) and moderate network resource allocation (10Mbps) will be applied.

[0091] For user terminal 1b, used by worker β performing the lowest-priority task E, the video compression rate will be increased to reduce transmission bandwidth, and the minimum network resource allocation (5Mbps) will be applied. In addition, all video except for high-priority tasks will be transmitted on a best-effort basis to ensure that fixed-bandwidth video reaches server 2 reliably.

[0092] Furthermore, this method anticipates that, as a control measure for high-priority tasks, it may not be limited to transmission of uncompressed video, but may also prioritize preventing video distortion and loss by increasing the video compression ratio. When ensuring the reach of video data, a method can be adopted to reduce the transmission rate by compressing the video while maintaining the highest possible resolution within the allocated communication bandwidth, thereby improving reach. In this way, the video quality control can be changed according to the system requirements and priorities, such as using uncompression when high resolution is important, or increasing the compression ratio to the extent possible when video data guarantee is important.

[0093] As described above, by coordinating network bandwidth control and video quality control based on task priorities, it becomes possible to ensure both high-definition video and stable communication.

[0094] Figure 11 is a flowchart showing the processing of Server 2 in the second embodiment. The processing unit 22 of Server 2 notifies each user terminal 1 of the initial values ​​of the transmission parameters and video quality parameters and starts acquiring the work video (S101).

[0095] Next, the video receiving unit 224 receives the video captured by the camera unit 11 of each user terminal 1 via network A (S102).

[0096] Next, the video analysis unit 225 analyzes the video, and the correct / incorrect determination unit 226 compares the video with the correct data to determine whether it is correct or incorrect (S103).

[0097] Next, the judgment result notification unit 227 notifies each user terminal 1 of the correct / incorrect judgment result (S104).

[0098] Furthermore, in S105, the video analysis unit 225 determines whether or not there has been a change in the work process based on the results of the video analysis. If the result is Yes, the process proceeds to S106; otherwise, the process proceeds to S109.

[0099] In S106, the network control unit 222 performs a network slicing reconfiguration process based on the priority of the tasks.

[0100] Next, in S107, it is determined whether the slicing update was successful or not. If Yes, proceed to S108; otherwise, proceed to S109.

[0101] In S108, the transmission control instruction unit 223 issues a transmission control instruction to each user terminal 1 to transmit video according to the changed communication bandwidth.

[0102] In S109, the processing unit 22 determines whether all operations have been completed. If the answer is yes, the process ends; otherwise, it returns to S102.

[0103] In this way, according to the second embodiment, it becomes possible to recognize changes in the work process and change the bandwidth control for a series of work videos from each user terminal 1, thereby achieving stable video transmission.

[0104] (Third embodiment) Next, a third embodiment will be described. Details similar to those described in at least one of the first and second embodiments will be omitted as appropriate.

[0105] Figure 12 is an overall configuration diagram of the video transmission system S of the third embodiment. As an example of a configuration different from the second embodiment, Figure 12 shows a system configuration that works in conjunction with overhead cameras 3 (3a, 3b).

[0106] For example, an overhead camera 3 is installed in each work area. In addition to the configuration shown in Figure 4, the server 2 also includes an area determination unit 228. The area determination unit 228 identifies the work area where each user terminal 1 is located from the video received from the overhead camera 3 and updates the work information database 211 with information about the work being performed by the worker using the user terminal 1.

[0107] Figure 13 is an explanatory diagram of the relationship between the overhead camera 3 and the work area in the third embodiment. Figure 13 shows an example of work priority determination based on area determination by the overhead camera 3. When the work content is determined for each work area, the system can recognize when a worker enters the work area to determine the work content and the start of the work, and recognize when the worker leaves the work area to determine the end of the work. Overhead camera 3a is photographing the work area for work process A, and overhead camera 3b is photographing the work area for work process B. The area determination unit 228 of the server 2, which receives the video from the overhead camera 3, recognizes that a worker using the user terminal 1 has entered the work area through image analysis.

[0108] While overhead camera 3a is capturing footage of a worker using user terminal 1a working in the work area (workbench 4A) of high-priority work process A, high-bandwidth network resources are allocated to user terminal 1a. On the other hand, if it is determined that the worker has left the work area of ​​overhead camera 3a and moved to the work area (workbench 4B) of lower-priority work process B, which is being captured by overhead camera 3b, the priority of user terminal 1a is changed from "high" to "low".

[0109] Furthermore, when a user terminal 1b used by a new worker enters the area of ​​the overhead camera 3a, it is determined that work process A has begun, and a high-priority, high-bandwidth network resource is allocated. In this way, when a work area corresponds to a work, using network control in conjunction with area determination means (area determination unit 228) makes it possible to determine the work priority more reliably.

[0110] In the example shown in Figure 13, one overhead camera 3 corresponds to one work area, but this is not limited to that. For example, one overhead camera 3 may capture an overhead view of multiple work areas. In that case, it would be possible to determine the priority of multiple tasks using the video from a single overhead camera 3 by determining the task content based on the position of the user terminal 1 as seen from the overhead camera 3.

[0111] Furthermore, it is possible to perform priority control by linking the location of the user terminal 1 with the work area using means of calculating the position, such as GPS (Global Positioning System), which is equipped on the user terminal 1, without using the overhead camera 3. In other words, the location detection method used for determination based on the work area is not limited.

[0112] The program executed by the video transmission system S of this embodiment can be provided as an installable or executable file recorded on a recording medium readable by a computer device, such as a CD (Compact Disc)-ROM (Read Only Memory), a flexible disk (FD), a CD-R (Recordable), or a DVD (Digital Versatile Disk). Alternatively, the program may be provided or distributed via a network such as the Internet.

[0113] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0114] For example, the network A to which the present invention applies is not limited to 5G (including local 5G), but may be 6G or later, or other communication networks, as long as they are communication networks capable of bandwidth allocation of network resources. [Explanation of symbols]

[0115] 1...User terminal, 2...Server, 3...Overhead camera, 4...Workbench, 11...Camera unit, 12...Video transmission unit, 13...Transmission control unit, 14...Display unit, 15...Judgment result receiving unit, 21...Storage unit, 22...Processing unit, 23...Input unit, 24...Output unit, 211...Work information database, 212...Video storage unit, 221...Network monitoring unit, 222...Network control unit, 223...Transmission control instruction unit, 224...Video receiving unit, 225...Video analysis unit, 226...Correct / incorrect judgment unit, 227...Judgment result notification unit, 228...Area judgment unit, S...Video transmission system

Claims

1. A network processing device connected to a user terminal used by a worker performing work at a designated work site, via a communication network capable of allocating network resource bandwidth, A network monitoring unit that detects network information including the bandwidth and utilization rate of the aforementioned communication network, A network control unit performs network control, including bandwidth control and bandwidth guarantee, on the communication network based on management information in a work information database that manages the priority of each task and the tasks being performed by the worker using the user terminal, and the network information. A transmission control instruction unit notifies the user terminal that transmits video footage of the work being done at the work site, based on the content of the network control, of transmission control information including at least one of the video compression ratio or transmission rate according to the priority of the work. A network processing device comprising: a video receiving unit that receives video transmitted from the user terminal based on the transmission control information and stores the received video in a video storage unit.

2. A video analysis unit analyzes the video using predetermined image processing and outputs analysis data, A correct / incorrect determination unit compares the correct data, which is a video of a sample operation, with the aforementioned analysis data to determine whether the operation is correct or incorrect and outputs the correct / incorrect determination result. The network processing device according to claim 1, further comprising a determination result notification unit that notifies the user terminal of the correct / incorrect determination result.

3. The aforementioned video storage unit stores video transmitted from multiple user terminals, The network processing apparatus according to claim 2, wherein the video analysis unit analyzes the video stored in the video storage unit by the image processing and outputs the analysis data.

4. The correct / incorrect determination unit outputs the correct / incorrect determination result, which includes whether or not there is an error in the work procedure. The network processing device according to claim 2, wherein the judgment result notification unit notifies the user terminal of the correct / incorrect judgment result as information for display.

5. The network processing device according to claim 1, wherein the network control unit determines, based on the network information, that the communication network bandwidth is insufficient, and allocates network bandwidth according to the priority of the work corresponding to the video.

6. The network control unit recognizes the tasks and priorities corresponding to the user terminal based on the management information in the work information database. If the priority is high, network control for high-quality real-time video will be implemented, including securing a fixed bandwidth or guaranteeing latency, at least one of the following: The network processing device according to claim 1, wherein, when the priority is low, network control is performed with best-effort bandwidth allocation.

7. The network processing apparatus according to claim 6, wherein the transmission control instruction unit notifies the user terminal to improve the video quality when the priority is high, and at that time performs video high-definition control, which includes improving the video resolution, improving the transmission rate, or decreasing the video compression ratio, in conjunction with the network control unit's network control for high-quality real-time video.

8. The network processing apparatus according to claim 6, wherein, when the transmission control instruction unit has a high priority, when performing transmission control to the user terminal, it performs transmission control in conjunction with the network control unit's network control for high-quality real-time video, including at least one of a reduction in resolution, a reduction in transmission rate, or an improvement in video compression ratio, within a range that ensures the necessary resolution of the video.

9. The network processing apparatus according to claim 6, wherein, when the transmission control instruction unit has a low priority, when performing transmission control to the user terminal, it performs transmission control in conjunction with the network control unit's best-effort bandwidth allocation network control, including at least one of reducing the video resolution, reducing the transmission rate, or improving the video compression ratio.

10. The network processing device according to claim 2, wherein the video analysis unit analyzes the video by the image processing and outputs the analysis data, identifies a task from the analysis data, and updates the task information database with information about the task being performed by the worker using the user terminal.

11. An overhead camera is installed in each work area. The aforementioned network processing device is The network processing device according to claim 2, further comprising: an area determination unit that identifies the work area where the user terminal is located from the video footage received from the overhead camera, and updates the work information database with information on the work being performed by the worker using the user terminal.

12. A video transmission method using a network processing device connected to a user terminal used by a worker performing work at a designated work site via a communication network capable of allocating network resource bandwidth, A network monitoring step in which the network monitoring unit detects network information including the bandwidth and utilization rate of the communication network, A network control step in which the network control unit performs network control, including bandwidth control and bandwidth guarantee, on the communication network based on management information in a work information database that manages the priority of each task and the tasks being performed by the worker using the user terminal, and the network information. A transmission control instruction step in which the transmission control instruction unit notifies the user terminal that transmits video footage of the work at the work site, based on the content of the network control, of transmission control information including at least one of the video compression ratio or transmission rate according to the priority of the work, A video transmission method comprising: a video receiving step in which a video receiving unit receives video transmitted from the user terminal based on the transmission control information, and stores the received video in a video storage unit.

13. A computer, which is a network processing device connected to a user terminal used by a worker performing work at a designated work site, via a communication network capable of allocating network resource bandwidth, A network monitoring unit that detects network information including the bandwidth and utilization rate of the aforementioned communication network, A network control unit performs network control, including bandwidth control and bandwidth guarantee, on the communication network based on management information in a work information database that manages the priority of each task and the tasks being performed by the worker using the user terminal, and the network information. A transmission control instruction unit notifies the user terminal that transmits video footage of the work being done at the work site, based on the content of the network control, of transmission control information including at least one of the video compression ratio or transmission rate according to the priority of the work. A program for causing a video receiving unit to function as a video receiving unit that receives video transmitted from the user terminal based on the transmission control information and stores the received video in a video storage unit.

Images