Relay device, relay method, relay program, and relay system

Abstract

To provide a relay device, a relay method, a relay program, and a relay system that improve the efficiency of communications.SOLUTION: A first path selection unit 14 selects a first communication path from among multiple communication paths between image capture devices on the basis of information about the image capture device. A first communication unit 11 acquires video data from an image capture device 20 using the first communication path selected by the first path selection unit 14. An AI processing unit 12 executes first processing using the video data acquired by the first communication unit 11. The second communication unit transmits the video data subjected to the first processing by the AI processing unit 12 to a video analysis server 3 using a second communication path.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a relay device, a relay method, a relay program, and a relay system. [Background technology]

[0002] In recent years, there has been an increase in the use of AI (Artificial Intelligence) cameras, which combine a camera or other imaging device for capturing video with a computer system for analyzing the captured video data. AI cameras are equipped with a GPU that performs high-speed computational processing such as deep learning-based AI, and a wireless or wired communication module. By using such AI cameras, processing such as feature detection and annotation from video data can be performed on the AI ​​camera side, which is expected to improve work efficiency.

[0003] One configuration of AI cameras is one in which video analysis processing is performed by a computer at the edge of the network. AI cameras that adopt this configuration include those equipped with an EAB (Edge Artificial-Intelligence Box), which processes video data using AI and transmits it via the network to a server where an application runs.

[0004] In such AI cameras, the EAB acquires video data from a camera connected to the user's network environment, and the results of AI processing within the EAB are sent via the Internet to a server, etc. However, since the camera and network environment differ for each user, it is desirable to carry out communication between the EAB and the camera in an optimal network environment based on information about the camera and the video generated.

[0005] As a video data distribution technology, a technology has been proposed in which a shooting device generates low-quality data from the same video signal, records the low-quality data in a management server, and records the high-quality data in a recording device of the shooting device, and switches between the low-quality data and the high-quality data and distributes them in response to user operation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-58994 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, image quality such as resolution and communication environment differ for each imaging device, and selecting an appropriate network environment for each imaging device is difficult because of the large workload involved in switching communication paths according to user instructions. In addition, if a communication path is selected that can handle high-quality data, network resources will be wasted in imaging devices that transmit low-quality data. Conversely, if a communication path that supports low-quality data is selected in order to maintain the network environment, the load on the imaging device that transmits high-quality data will be high, and communication itself may become difficult. As described above, with conventional relay technology, there is a risk of wasting network resources and increasing network load, making it difficult to improve communication efficiency.

[0008] The disclosed technology has been made in consideration of the above, and aims to provide a relay device, a relay method, a relay program, and a relay system that improve the efficiency of communications. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the object, the relay device of the present invention has the following units. A first route selection unit selects a first communication route from among a plurality of communication routes between the relay device and the image capture device based on information about the image capture device. A first communication unit acquires video data from the image capture device using the first communication route selected by the first route selection unit. A processing unit executes a first processing using the video data acquired by the first communication unit. A second communication unit transmits the video data that has been subjected to the first processing by the processing unit to a data processing device using the second communication route. Effect of the Invention

[0010] According to the present invention, it is possible to improve the efficiency of communication. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a system configuration diagram of a relay system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a connection between the EAB and an imaging device via a customer LAN. [Diagram 3] FIG. 3 is a diagram showing an example of a direct connection of the EAB to an imaging device. [Figure 4] FIG. 4 is a block diagram of an EAB according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of communication path selection. [Figure 6] FIG. 6 is a diagram showing an example of communication path selection accompanied by a change in resolution setting. [Figure 7] FIG. 7 is a diagram showing communication settings of the EAB in response to a connection with a video analysis server. [Figure 8] FIG. 8 is a flowchart of a communication path determination process by the EAB according to the embodiment. [Figure 9] FIG. 9 is a diagram showing the hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the relay device, relay method, relay program, and relay system disclosed in the present application will be described in detail with reference to the drawings. Note that the relay device, relay method, relay program, and relay system disclosed in the present application are not limited to the following embodiments.

[0013] (Embodiment) (Introduction) The system acquires video from a camera connected to the client's network environment, analyzes the video within the device, and transmits the results via the Internet. The client's network environment has already been constructed and is in operation, and there are concerns about the impact of adding new video analysis data. Therefore, it was decided to use a relay device to perform multiple simultaneous communications with different communication types. However, since the resolution of each imaging device is different, it is difficult to automatically perform communication between the relay device and the imaging device in the optimal network environment.

[0014] Therefore, the relay device according to the embodiment prioritizes and classifies communication paths in consideration of the amount of data to be communicated and information that can be acquired from the image capture device of the communication destination, and determines the communication path to be used without requiring user operation. Details of the relay device according to the embodiment will be described below.

[0015] (System Configuration) 1 is a system configuration diagram of a relay system according to an embodiment. The relay system 100 includes an EAB 1 which is a relay device, a customer LAN (Local Area Network) 2, a video analysis server 3 on a cloud, and a terminal device 4.

[0016] In the customer LAN 2, there are arranged IP (Internet Protocol) cameras 21 to 23 which are image capturing devices, a customer terminal device 201, a hub 202, etc. In addition, to the customer LAN 2, an EAB 1 is connected.

[0017] The IP cameras 21-23 are connected to the EAB1 via the customer LAN 2. The IP cameras 21-23 are connected to the EAB1 via a plurality of communication paths. The IP cameras 21-23 may be connected to the EAB1 via a hub 202. For example, the IP cameras 21-23 can communicate with the EAB1 using a wired connection such as Ethernet (registered trademark), a wireless connection such as Wi-Fi, or a wireless connection such as private 5G. However, these communication paths are merely examples, and other communication paths may be used as long as they comply with a standard that allows data to be transferred.

[0018] The specifications for shooting and data transfer are determined in advance for each of the IP cameras 21-23. For example, the resolution, image size, frame rate (fps: Frames Per Second), and video data codec that can be used are determined in advance for the IP cameras 21-23. Here, each of the IP cameras 21-23 may be able to use multiple types for each setting. For example, the IP camera 21 may be able to use multiple resolutions.

[0019] The settings used for image capture and data transfer are designated in advance by an administrator for the IP cameras 21 to 23. The IP cameras 21 to 23 hold image capture device information in which the resolution, image size, frame rate, codec, etc. that each image capture device 20 can use are registered in addition to the manufacturer name and model name.

[0020] Each of the IP cameras 21-23 captures video according to the specified settings. When the IP cameras 21-23 receive an instruction from the EAB1 to change the resolution setting, they change the setting to the specified resolution and capture video. Then, the IP cameras 21-23 transmit the video data of the captured video to the EAB1 using the communication path specified by the EAB1.

[0021] Although the IP cameras 21 to 23 have been described as an example of the imaging devices here, the imaging devices may be other devices that capture images.

[0022] The customer terminal device 201 can be connected to the IP cameras 21 to 23 via a hub 202 or the like. The administrator of the IP cameras 21 to 23 can use the customer terminal device 201 to configure the settings of the IP cameras 21 to 23.

[0023] The EAB 1 is connected to IP cameras 21 to 23 via a customer LAN 2. Fig. 2 is a diagram showing an example of a connection between the EAB and an imaging device via the customer LAN 2. For example, the EAB 1 has a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103, as shown in Fig. 2.

[0024] The EAB1 can communicate with the IP cameras 21-23 via the customer LAN 2 using one or more of the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. For example, in FIG. 2, the EAB1 is connected to the customer LAN 2 using the Wi-Fi communication interface 103. In this case, the EAB1 can communicate with the IP cameras 21-23 using a communication path via Wi-Fi communication.

[0025] 2 shows a state in which the EAB 1 is connected via one path, the EAB 1 may be connected to the customer LAN 2 via multiple interfaces, among the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. In this case, the EAB 1 can select a communication path from the connected paths to communicate with the IP cameras 21 to 23.

[0026] Fig. 3 is a diagram showing an example of a direct connection between an EAB and an imaging device. For example, the EAB1 may be directly connected to an IP camera 21 via a network as shown in Fig. 3. In Fig. 3, the EAB1 is connected to the IP camera 21 using a wired communication interface 102. However, the EAB1 may be connected to the IP camera 21 via a communication path using another interface depending on the network function of the IP camera 21, or may be connected via multiple communication paths.

[0027] Returning to FIG. 1, the explanation continues. The EAB1 specifies the type of communication path that is optimal for transmitting and receiving video data for each of the IP cameras 21-23 from among a plurality of communication paths connecting the IP cameras 21-23, according to the resolution of the video generated by each of the IP cameras 21-23. The EAB1 then notifies each of the IP cameras 21-23 of the specified type of communication path as the communication path to be used for transmitting the video data. If the specified type of communication path is unavailable, the EAB1 determines an available communication path that corresponds to a lower resolution as the communication path to be used. The EAB1 then instructs the IP cameras 21-23 to change the settings to transmit video data via the communication path that has been determined to be used, and notifies the IP cameras 21-23 of the communication path to be used for transmitting the video data.

[0028] Furthermore, the EAB1 executes AI processing using a designated AI application for each piece of video data transmitted from the IP cameras 21 to 23. Here, the EAB1 can use the same AI application as the AI ​​application for processing each piece of data, or can use different applications.

[0029] Furthermore, the EAB1 determines, according to the designated priority, a communication path to be used for transmitting the video data and the result of the AI ​​processing from among a plurality of communication paths connecting with the video analysis server 3 on the cloud. Then, the EAB1 transmits the result of the AI ​​processing and the video data to the video analysis server 3 using the determined communication path. The EAB1 will be described in detail later.

[0030] The video analysis server 3 is a device deployed on the cloud. The video analysis server 3 is connected to the EAB 1 via the Internet. The video analysis server 3 receives the AI ​​processing result and the video data from the EAB 1. The video analysis server 3 then performs a process of adding added value to the video data, for example, by visualizing the AI ​​processing result. The video analysis server 3 then transmits the processed video data to the terminal device 4. This video analysis server 3 is an example of a "data processing device," and the process by the video analysis server 3 using the video data and the AI ​​processing result is an example of a "second process."

[0031] The terminal device 4 is a device that performs various controls and the like using the images captured by the IP cameras 21 to 23. For example, the terminal device 4 executes control processing for realizing smart retail, smart buildings, smart cities, smart factories, and the like, using the images captured by the IP cameras 21 to 23.

[0032] (EAB configuration) Fig. 4 is a block diagram of an EAB according to an embodiment. Next, with reference to Fig. 4, the EAB 1 according to this embodiment will be described in detail. Here, each device for generating an image, such as IP cameras 21 to 23, will not be distinguished from the others, and will be described as an image capture device 20. As shown in Fig. 2, the EAB 1 has a first communication unit 11, an AI processing unit 12, a second communication unit 13, a first route selection unit 14, and a second route selection unit 15.

[0033] The first communication unit 11 is connected to one or more image capture devices 20 via multiple communication paths. The multiple communication paths between the first communication unit 11 and each image capture device 20 are an example of the "first multiple communication paths." For example, the first communication unit 11 has a communication path for wired connection by Ethernet communication, a communication path for wireless connection by Wi-Fi communication, and a communication path for wireless connection by private 5G communication. However, the communication paths available between the first communication unit 11 and each image capture device 20 do not have to be the same.

[0034] The first communication unit 11 receives the camera device information including information on the image quality of the video data generated by the camera device 20. For example, when the EAB1 is connected to the customer LAN 2, the first communication unit 11 obtains the camera device information in which information on settings such as resolution is registered from each camera device 20 connected to the customer LAN 2 using any of the available communication paths.

[0035] Then, the first communication unit 11 outputs the image capture device information to the first path selection unit 14. After that, the first communication unit 11 receives information on a communication path to be used by the image capture device 20 in transmitting video data from the first path selection unit 14, and transmits the information to the image capture device 20.

[0036] Furthermore, when the first communication unit 11 receives an input of an instruction to change the resolution setting from the first path selection unit 14, it transmits the instruction to change the resolution setting to the targeted image capture device 20, thereby changing the resolution setting of the image capture device 20. Then, the first communication unit 11 receives information on a communication path corresponding to the changed resolution from the first path selection unit 14 as information on a communication path to be used by the image capture device 20 in transmitting video data, and transmits the information to the image capture device 20.

[0037] Thereafter, the first communication unit 11 receives the video data from the image capturing device 20 via the communication path designated by the first path selection unit 14. Then, the first communication unit 11 outputs the received video data to the AI ​​processing unit 12.

[0038] The first path selection unit 14 receives an input of the imaging device information for each imaging device 20 from the first communication unit 11. In addition to the manufacturer name and model name, the imaging device information registers information indicating image quality such as resolution, image size, frame rate, and codec that each imaging device 20 can use. The first path selection unit 14 performs the following path selection process for each imaging device 20. Since the path selection process for each imaging device 20 is performed in the same way, the path selection process will be described below using one imaging device 20 as an example.

[0039] The first path selection unit 14 acquires the resolution of the imaging device 20 from the imaging device information. Here, the first path selection unit 14 holds information on the type of communication path to be used for each resolution. For example, the first path selection unit 14 divides the resolution into three sections, high, medium, and low, according to the height, and holds information on the type of communication path to be used for each section. Specifically, the first path selection unit 14 holds information such as using a communication path by wired communication for the resolution in the high section, using a communication path by private 5G communication for the resolution in the medium section, and using a communication path by Wi-Fi communication for the resolution in the low section. However, the division of the resolution is not limited to three stages, and may be two stages or four or more stages, and the type of communication path corresponding to each resolution may be determined.

[0040] The first path selection unit 14 identifies the type of communication path according to the resolution of the image capture device 20 acquired from the image capture device information. Next, the first path selection unit 14 determines whether or not communication with the image capture device 20 is possible via the identified type of communication path. For example, when the identified type of communication path does not exist between the image capture device 20 and the image capture device 20, or when the identified type of communication path exists but is difficult to use due to a failure, the first path selection unit 14 determines that communication with the image capture device 20 is not possible via the identified type of communication path.

[0041] If communication is possible, the identified type of communication path is determined as the communication path to be used for transmitting and receiving video data from the image capture device 20. After that, the first path selection unit 14 notifies the image capture device 20 via the first communication unit 11 of information on the communication path to be used by the image capture device 20 in transmitting the video data.

[0042] 5 is a diagram showing an example of communication path selection. For example, a case will be described in which the EAB1 is connected to each of the IP cameras 21 to 23 via a communication path 211 for wired communication, a communication path 212 for Wi-Fi communication, and a communication path 213 for private 5G communication. Also, a case will be described in which the first path selection unit 14 divides resolution into three sections, high, medium, and low, and uses the communication paths 211, 213, and 212 in order from the section with the highest resolution.

[0043] The first path selection unit 14 confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. Then, the first path selection unit 14 specifies the communication path 211 by wired communication corresponding to the high category as an appropriate communication path for transmitting and receiving video data with the IP camera 21. Also, the first path selection unit 14 confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. Then, the first path selection unit 14 specifies the communication path 213 by private 5G communication corresponding to the medium category as an appropriate communication path for transmitting and receiving video data with the IP camera 22. Also, the first path selection unit 14 confirms from the imaging device information of the IP camera 23 that the IP camera 23 uses a resolution included in the low category. Then, the first path selection unit 14 specifies the communication path 212 by Wi-Fi communication corresponding to the low category as an appropriate communication path for transmitting and receiving video data with the IP camera 23. Since communication with the IP cameras 21 to 23 is possible via any of the communication paths, the first path selection unit 14 determines each of the communication paths as the communication path to be used. As a result, the first communication unit 11 communicates with the IP cameras 21 to 23 using the communication paths corresponding to the respective resolutions, as shown in FIG.

[0044] Returning to Fig. 4, the explanation will be continued. On the other hand, when communication is not possible through the specified type of communication path, for example, because the specified type of communication path is not connected to the image capture device 20 or is unavailable due to a fault, the first path selection unit 14 determines whether communication with the image capture device 20 is possible through a communication path corresponding to a lower resolution. Specifically, the first path selection unit 14 determines whether communication with the image capture device 20 is possible through a communication path corresponding to a lower resolution, and whether the image capture device 20 supports that resolution.

[0045] When a communication path corresponding to a lower resolution is available, the first path selection unit 14 determines the communication path to be used as the communication path having the highest possible resolution among the resolutions corresponding to the available communication paths that the camera device 20 can use. This communication path is a communication path corresponding to the highest possible resolution among the resolutions corresponding to the communication paths that the EAB1 can use between the camera device 20. Next, the first path selection unit 14 transmits an instruction to the camera device 20 via the first communication unit 11 to change the setting to the highest possible resolution that the camera device 20 can use among the resolutions corresponding to the determined communication path. After that, the first path selection unit 14 notifies the camera device 20 via the first communication unit 11 of information on the communication path to be used by the camera device 20 in transmitting video data.

[0046] 6 is a diagram showing an example of communication path selection accompanied by a change in resolution setting. For example, a case will be described in which the EAB1 is connected to each of the IP cameras 21 to 23 via a communication path 212 by Wi-Fi communication, and the communication path 211 by wired communication and the communication path 213 by private 5G communication cannot be used due to a failure or the like. Here, too, the first path selection unit 14 divides the resolution into three sections, high, medium, and low, and uses the communication paths 211, 213, and 212 in order from the section with the highest resolution.

[0047] In this case, the first path selection unit 14 also confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. Then, the first path selection unit 14 judges whether the communication path 211 corresponding to the high category is available. In this case, the first path selection unit 14 judges that the communication path 211 cannot be used and that the communication path 212 by Wi-Fi communication is available. Then, the first path selection unit 14 decides to use the communication path 212 using Wi-Fi corresponding to the low category for transmitting and receiving video data to and from the IP camera 21. After that, the first path selection unit 14 instructs the IP camera 21 to change the setting to a resolution that the IP camera 21 can use and that is as high as possible, among the resolutions that can be used on the communication path 212. Also, the first path selection unit 14 confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. Then, the first path selection unit 14 judges whether the communication path 213 corresponding to the medium category is available. In this case, the first path selection unit 14 determines that the communication path 213 cannot be used, and that the communication path 212 by Wi-Fi communication is available. Then, the first path selection unit 14 decides to use the communication path 212 by Wi-Fi communication corresponding to the low category for transmitting and receiving video data to and from the IP camera 22. After that, the first path selection unit 14 instructs the IP camera 22 to change the setting to a resolution that is usable by the IP camera 22 and is as high as possible among the resolutions that can be used on the communication path 212. Also, the first path selection unit 14 confirms that the IP camera 23 uses a resolution included in the low category from the imaging device information of the IP camera 23. In this case, since the communication path 211 using Wi-Fi corresponding to the low category is usable, the first path selection unit 14 decides to use the communication path 211 for transmitting and receiving video data to and from the IP camera 23. Then, the first path selection unit 14 communicates with all of the IP cameras 21 to 23 using the communication path 212, as shown in FIG. 5.

[0048] For example, the communication path 211 according to the resolution of the IP camera 21 is an example of the "first communication path", and the inability to communicate between the IP camera 21 and the EAB1 via the first communication path is an example of the "case where it is difficult to select the first communication path". Furthermore, the communication path 212 selected by the first path selection unit 14 other than the communication path 211 suitable for the resolution of the IP camera 21 is an example of the "third communication path". That is, when it is difficult to select the communication path 211, which is the first communication path based on the information of the IP camera 21, the first path selection unit 14 selects the communication path 212, which is the third communication path other than the communication path 211 suitable for the resolution of the IP camera 21, from among the communication paths between the image capture device 20, and changes the settings of the IP camera 21 so as to transmit video data via the third communication path.

[0049] In this embodiment, the different types of communication paths 211 to 213 are associated with the different resolution classes on a one-to-one basis, and the first path selection unit 14 selects a communication path corresponding to the resolution. In addition, the first path selection unit 14 changes the settings of the image capture device 20 so that the highest possible resolution can be used among the available communication paths in order to improve the image quality of the video data. However, the method of selecting a communication path for a resolution and changing the resolution settings of the image capture device 20 is not limited to this.

[0050] For example, the resolution classifications can be associated with the communication paths 211 to 213 as follows. The communication path 211 using wired communication can transmit video data in all resolution classifications, high, medium, and low. The communication path 212 using private 5G communication can transmit video data in two resolution classifications, medium and low. In contrast, the communication path 213 using Wi-Fi communication can transmit only video data in the low resolution classification. In this case, the first path selection unit 14 may determine the communication path used by the image capture device 20 to transmit video data as follows.

[0051] For example, when the resolution of the image capturing device 20 is included in the low category, all of the communication paths 211 to 213 are available, and the network resources of the communication path 211 via wired communication are greater than those of the communication paths 212 and 213, the first path selection unit 14 may use the communication path 211 via wired communication for transmitting video data of the image capturing device 20 in order to effectively use the network resources. In this case, the selection of the communication path 211 based on the resolution and network resources of the image capturing device 20 is an example of "selecting a first communication path based on information about the image capturing device".

[0052] In this case, if multiple image capture devices 20 with low resolution are connected to the EAB1 and all of the communication paths 211 to 213 are available, connecting all of the image capture devices 20 to the communication path 211 by wired communication may cause a load on the network. Therefore, when the communication speed of the communication path 211 becomes equal to or lower than a threshold, the first path selection unit 14 may change the communication path 212 by private 5G communication to transmit video data of some of the image capture devices 20 so that the communication speed falls within the threshold. Similarly, when the communication speed of the communication path 212 becomes equal to or lower than a threshold, the first path selection unit 14 may change the communication path 212 to use the communication path 213 by Wi-Fi communication to transmit video data of some of the image capture devices 20 so that the communication speed falls within the threshold.

[0053] The same applies when the resolution of the image capture device 20 is included in the high category, and when the communication speed of the communication path 211 falls below a threshold, the first path selection unit 14 may change to use the communication path 212 by private 5G communication for transmitting video data of some image capture devices 20 so that the communication speed falls within the threshold. In this case, the first path selection unit 14 instructs the target image capture device 20 to change the setting to a resolution in the medium category in order to use the communication path 212 by private 5G communication.

[0054] The case where the communication speed becomes equal to or lower than the threshold is an example of a case where it is difficult to select the first communication path. The communication path 212 selected instead of the communication path 211 is an example of a third communication path.

[0055] In this embodiment, the first path selection unit 14 determines the communication path according to the resolution that defines the amount of video data generated by the imaging device 20. However, the criteria for defining the amount of data to determine the communication path are not limited to this, and other information can be used as long as it is information that can grasp the amount of data to be transmitted. For example, the amount of data can also be defined by other information that indicates the image quality of the video, such as image size, frame rate (FPS), bit rate, or codec. Therefore, the first path selection unit 14 may obtain such information that indicates the image quality of the video from the imaging device information and determine the communication path to be used.

[0056] Here, the first route selection unit 14 may identify these pieces of information from the manufacturer name and model name of the photographing device 20 registered in the photographing device information. Also, the first route selection unit 14 may combine some of these pieces of information and use them to determine the communication route.

[0057] Furthermore, the first path selection unit 14 may acquire radio wave strength in wireless communication with the image capture device 20 and use it for selecting a communication path, such as not using wireless communication with a radio wave strength below a reference value. Additionally, the first path selection unit 14 may use the type of AI application corresponding to the image capture device 20, which is used for AI processing in the AI ​​processing unit 12 for each piece of video data, for selecting a communication path.

[0058] For example, when receiving video data to be processed by an AI application that can tolerate a small drop in data, the first path selection unit 14 can select a communication path that can transfer a small amount of data. In other words, the first path selection unit 14 can select a communication path based on the video quality required for the AI ​​application to maintain a minimum level of performance.

[0059] In this way, the first path selection unit 14 can determine a communication path to be used for transmitting video data between the image capture device 20, based on various information about the image capture device 20, such as information indicating the image quality, radio wave intensity, and the type of AI application. Here, the first path selection unit 14 selects a communication path corresponding to the information acquired based on a predetermined correspondence relationship, but is not limited to this and may determine a communication path using, for example, AI. The communication path selected by the first path selection unit 14 to be used for transmitting video data between the first communication unit 11 and the image capture device 20 is an example of a "first communication path."

[0060] Returning to FIG. 4, the explanation will be continued. The AI ​​processing unit 12 receives an input of video data transmitted from each of the multiple image capture devices 20 from the first communication unit 11. The AI ​​processing unit 12 has multiple AI applications. For example, one AI application may be an application for detecting a human face, and another AI application may be an application for detecting an automobile, and each AI application may execute a different AI process.

[0061] The AI ​​processing unit 12 executes AI processing for each acquired video data using a designated AI application. The AI ​​processing unit 12 may hold information predetermined for each image capture device 20 as the designation of the AI ​​application to be used, or may acquire the designation of the application to be used from information added to the video data. The AI ​​processing unit 12 then outputs the result of the AI ​​processing together with the video data to the second communication unit 13. This AI processing unit 12 is an example of a "processing unit", and the AI ​​processing of the video data by the AI ​​processing unit 12 is an example of a "first processing".

[0062] The second path selection unit 15 receives an input of a priority order designation for each of the multiple communication paths connecting the second communication unit 13 and the video analysis server 3 from an input unit (not shown). The multiple communication paths connecting the second communication unit 13 and the video analysis server 3 are an example of a "second multiple communication paths". The second path selection unit 15 then determines the communication path with the highest priority among the available communication paths as the communication path to be used for communication with the video analysis server 3. The communication path selected by the second path selection unit 15 to be used for transmitting video data and AI processing results between the second communication unit 13 and the video analysis server 3 is an example of a "second communication path".

[0063] For example, a case will be described in which the EAB1 has a communication path 301 by a wired connection using Ethernet, a communication path 302 by 5G communication, a communication path 303 by LTE (Long Term Evolution) communication, and a communication path 304 by Wi-Fi, as shown in Fig. 5. The second path selection unit 15 receives a designation in the order of decreasing priority, for example, the communication path 301, the communication path 302, the communication path 303, and the communication path 304. In this case, the second path selection unit 15 determines the communication path 301 by a wired connection, which has the highest priority among the available communication paths, as the communication path to be used for communication with the video analysis server 3. As a result, the EAB1 communicates with the video analysis server 3 using the communication path 301 by wired connection.

[0064] Returning to Fig. 4, the explanation will be continued. The second path selection unit 15 instructs the second communication unit 13 to communicate with the video analysis server 3 using the communication path determined to be used for communication. Then, the second path selection unit 15 sets the communication path determined to be used for communication with the video analysis server 3 to ON and sets other communication paths to OFF in the communication settings of the second communication unit 13. However, when the first communication unit 11 and the second communication unit 13 communicate using the same communication interface, the second path selection unit 15 maintains the ON setting for the communication path used by the first communication unit 11.

[0065] 7 is a diagram showing communication settings of the EAB 1 in response to a connection with a video analysis server. Here, the EAB 1 is described as having a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103 as communication interfaces. It is also assumed that the wired communication interface 102 is used for communication between the EAB 1 and the customer LAN 2.

[0066] A connection state 111 indicates a state in which the EAB 1 and the video analysis server 3 communicate with each other via mobile data communication. In this case, the second path selection unit 15 turns on the setting for communication via the mobile data communication interface 101, leaves the setting for communication via the wired communication interface 102 on, and turns off the setting for communication via the Wi-Fi communication interface 103.

[0067] A connection state 112 indicates a state in which the EAB 1 and the video analysis server 3 communicate with each other via Wi-Fi. In this case, the second path selection unit 15 turns off the setting for communication via the mobile data communication interface 101, leaves the setting for communication via the wired communication interface 102 on, and turns on the setting for communication via the Wi-Fi communication interface 103.

[0068] The connection state 112 indicates a state in which the EAB 1 and the video analysis server 3 communicate with each other via wired communication. In this case, the second path selection unit 15 turns off the communication settings via the wired communication interface 102 and the Wi-Fi communication interface 103, while leaving the communication setting via the wired communication interface 102 on.

[0069] Returning to Fig. 4, the explanation will be continued. The second communication unit 13 is connected to the video analysis server 3 via one or more communication paths. For example, the second communication unit 13 can have a communication path for a wired connection using Ethernet communication, a communication path for a wireless connection using 5G communication, a communication path for wireless communication using LTE communication, and a communication path for a wireless connection using Wi-Fi.

[0070] The second communication unit 13 receives designation of a communication path to be used for communication with the video analysis server 3 from the second path selection unit 15. Then, the second communication unit 13 transmits the video data and the results of the AI ​​processing to the video analysis server 3 using the communication path designated by the second path selection unit 15. At this time, the second communication unit 13 can transmit each of the video data and the results of the AI ​​processing to the video analysis server 3 together using the same communication path.

[0071] (Relay processing) 8 is a flowchart of a process of determining a communication path by the EAB 1 according to the embodiment. Next, a flow of a process of determining a communication path by the EAB 1 according to the embodiment will be described with reference to FIG.

[0072] The first route selection unit 14 acquires the photographing apparatus information of each of the photographing apparatuses 20 connected to the customer LAN 2 (step S1).

[0073] Next, the first route selection unit 14 selects one image capturing device 20 (step S2).

[0074] Next, the first route selection unit 14 identifies a communication route suitable for transmitting and receiving video data to and from the selected camera device 20 based on the acquired camera device information (step S3).

[0075] Next, the first route selection unit 14 determines whether the identified communication route exists as a communication route connecting with the image capture device 20 and is usable as a communication route to be used for transmitting and receiving video data (step S4).

[0076] If the identified communication path is available for use (step S4: Yes), the first path selection unit 14 determines the identified communication path as the communication path to be used (step S5). After that, the communication path determination process proceeds to step S8.

[0077] On the other hand, if it is difficult to use as a communication path to be used (step S4: No), the first path selection unit 14 extracts a resolution that is as high as possible from among the resolutions corresponding to the available communication paths between the selected image capture device 20 and the image capture device 20 that can be used by the image capture device 20. Then, the first path selection unit 14 determines the communication path corresponding to the extracted resolution from among the available communication paths as the communication path to be used (step S6).

[0078] Then, the first route selection unit 14 notifies the image capture device 20 via the first communication unit 11 of an instruction to change the setting to the resolution corresponding to the determined communication route, and changes the setting of the image capture device 20 (step S7). After that, the communication route determination process proceeds to step S8.

[0079] Thereafter, the first route selection unit 14 notifies the image capturing device 20 of the determined communication route to be used (step S8).

[0080] Next, the first route selection unit 14 judges whether the determination of the communication route has been completed for all the photographing devices 20 connected to the customer LAN 2 (step S9). If there are any photographing devices 20 for which the communication route has not been determined (step S9: No), the communication route determination process returns to step S2.

[0081] On the other hand, if the determination of the communication paths for all the image capture devices 20 has been completed (step S9: Yes), the first path selection unit 14 completes the setting of the communication paths with the image capture devices 20. Next, the second path selection unit 15 acquires from the input unit the priority of the communication paths used for transmitting and receiving the video data and the results of the AI ​​processing with the video analysis server 3 (step S10).

[0082] Thereafter, the second route selection unit 15 determines a communication route to be used with the video analysis server 3 according to the priority. Then, the second route selection unit 15 performs communication settings of the second communication unit 13 so that video data is transmitted and received with the video analysis server 3 using the determined communication route (step S11).

[0083] The first communication unit 11 receives the video data from each of the image capture devices 20 via the determined communication path between the first communication unit 11 and each of the image capture devices 20 (step S12).

[0084] The AI ​​processing unit 12 performs AI processing on the video data received by the first communication unit 11 (step S13).

[0085] The second communication unit 13 transmits the video data received from each image capture device 20 and the results of each AI processing to the video analysis server 3 via the determined communication path between the second communication unit 13 and the video analysis server 3 (step S13).

[0086] (effect) As described above, the EAB1 according to this embodiment determines the type of communication path to be used based on the data volume determined by the image quality, such as the resolution, of the video data generated by the image capture device 20. This makes it possible to select the optimal communication path for each image capture device 20, automatically suppressing the waste of network resources and the increase in network load, and improving communication efficiency.

[0087] Furthermore, when the optimal communication path obtained from the amount of video data to be sent cannot be used, the EAB1 according to this embodiment changes the settings of the imaging device 20 to reduce the amount of video data, and selects a suitable communication path from among the communication paths with reduced data amounts as the second best communication path. This makes it possible to improve communication efficiency by minimizing waste of network resources and increases in network load, even when the communication path considered to be optimal cannot be used.

[0088] That is, the EAB1, which is a relay device according to the embodiment, has a first route selection unit 14 that selects a first communication route from among a first plurality of communication routes between the camera device 20 based on information about the camera device 20, a first communication unit 11 that acquires video data from the camera device 20 using the first communication route selected by the first route selection unit 14, an AI processing unit 12 that performs first processing using the video data acquired by the first communication unit 11, and a second communication unit 13 that transmits the video data subjected to the first processing by the AI ​​processing unit 12 to the data processing device using the second communication route.

[0089] This allows the EAB 1 to automatically select the optimal communication path to receive video data from the imaging device 20, automatically preventing waste of network resources and an increase in network load, and improving communication efficiency.

[0090] In the EAB1 which is a relay device according to the embodiment, the first route selection unit 14 selects the first communication route based on information defining the amount of video data included in the information of the image capturing device 20.

[0091] This allows the EAB 1 to select an appropriate communication path depending on the amount of data, thereby ensuring efficient communication.

[0092] In the EAB1 which is a relay device according to the embodiment, the first path selection unit 14 selects the first communication path based on information indicating image quality as information defining the data amount of video data.

[0093] This allows the EAB 1 to select an appropriate communication path according to the amount of data using image quality as a criterion, thereby making it possible to more reliably improve communication efficiency.

[0094] The EAB1, which is a relay device in the embodiment, further includes a second path selection unit 15 that selects a second communication path based on the priority of each of the second multiple communication paths between the EAB1 and the video analysis server 3, which is a data processing device, and the second communication unit 13 transmits the video data using the second communication path selected by the second path selection unit 15.

[0095] This enables the EAB1 to automatically select the optimal communication route and send video data and AI processing results to the video analysis server 3, automatically reducing the waste of network resources and the increase in network load, thereby improving communication efficiency.

[0096] (program) It is also possible to create a program in which the processing performed by the EAB1 described in the above embodiment is written in a language executable by a computer. In this case, the same effects as those of the above embodiment can be obtained by having the computer execute the program. Furthermore, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer to realize the same processing as those of the above embodiment.

[0097] Fig. 9 is a hardware configuration diagram of a computer. The EAB 1 is realized, for example, by a computer 90 shown in Fig. 9. The computer 90 has, for example, a processor 91, a memory 92, a hard disk 93, and a communication interface 94 as shown in Fig. 9.

[0098] The processor 91 operates based on programs stored in the hard disk 93 and controls each part. The memory 92 stores, for example, a boot program executed by the processor 91 when the computer 90 is started up. The hard disk 93 stores the programs executed by the processor 91 and data used by the programs.

[0099] The communication interface 94 receives data from other devices via a communication network NW (Network) and sends the data to the processor 91, and transmits data generated by the processor 91 to other devices via the communication network NW. The communication interface 94 realizes the functions of the first communication unit 11 and the second communication unit 13 illustrated in FIG.

[0100] The processor 91 reads out various programs stored in the hard disk 93, expands them in the memory 92, and executes them to realize the functions of the AI ​​processing unit 12, the first path selection unit 14, and the second path selection unit 15 illustrated in Fig. 4. The processor 91 reads out and executes programs from the hard disk 93, but as another example, these programs may be obtained from other devices via the communication network NW.

[0101] (others) Although various embodiments have been described in detail herein with reference to the drawings, these embodiments are merely examples and are not intended to limit the present invention to these embodiments. The features described herein can be realized in various ways, including various modifications and improvements based on the knowledge of those skilled in the art.

[0102] Furthermore, the above-mentioned "module (-er suffix, -or suffix)" can be read as a unit, a means, a circuit, a step, etc. For example, a communication module, a processing module, and a selecting module can be read as a communication unit, a control unit, and a storage unit, respectively. [Explanation of symbols]

[0103] 1 EAB 2 Customer LAN 3. Video analysis server 4 Terminal Equipment 11 First Communications Department 12 AI Processing Unit 13 2nd Communications Department 14 First Route Selection Section 15 Second Route Selection Section 20 Imaging Equipment 21~23 IP Camera 100 Relay System 201 Customer terminal equipment 202 Hub

Claims

1. a first route selection unit that selects a first communication route from among a plurality of communication routes between the image capture device and the image capture device based on information about the image capture device; a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit; a processing unit that executes a first process using the video data acquired by the first communication unit; a second communication unit that transmits the video data that has been subjected to the first processing by the processing unit to a data processing device via a second communication path; A relay device comprising:

2. The relay device according to claim 1 , wherein the first route selection unit selects the first communication route based on information defining a data amount of the video data included in the information of the imaging device.

3. 3. The relay device according to claim 2, wherein the first path selection unit selects the first communication path based on information representing image quality as information defining the data amount of the video data.

4. a second path selection unit that selects the second communication path based on the priority of each of a plurality of communication paths between the data processing device and the second communication path selection unit; The second communication unit transmits the video data using the second communication path selected by the second path selection unit.

2. The relay device according to claim 1 .

5. The relay device according to claim 1 , wherein the first route selection unit transmits a setting change instruction to the image capture device.

6. when it is difficult to select the first communication path from among the plurality of communication paths based on the information of the image capture device, the first path selection unit selects a third communication path other than the first communication path based on the information of the image capture device from among the plurality of communication paths, and changes a setting of the image capture device so as to transmit video data via the third communication path; The first communication unit acquires video data from an image capturing device using the third communication path selected by the first path selection unit.

2. The relay device according to claim 1 .

7. a first route selection step of selecting a first communication route from among a plurality of communication routes between the image capture device and the image capture device based on information of the image capture device; a first communication step of acquiring video data from an image capturing device using the first communication path selected by the first path selection step; a processing step of executing a first process using the video data acquired by the first communication step; a second communication step of transmitting the video data subjected to the first processing by the processing step to a data processing device via a second communication path; A relay method comprising:

8. a first route selection step of selecting a first communication route from among a plurality of communication routes between the image capture device and the image capture device based on information of the image capture device; a first communication step of acquiring video data from an image capturing device using the first communication path selected by the first path selection step; a processing step of performing a first process using the video data acquired by the first communication step; a second communication step of transmitting the video data subjected to the first processing by the processing step to a data processing device via a second communication path; A relay program that causes a computer to execute the above steps.

9. A relay system having an imaging device, a relay device, and a data processing device, the image capturing device is connected to the relay device via a plurality of communication paths; The relay device is a first route selection unit that selects a first communication route from the plurality of communication routes based on information about the image capturing device; a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit; a processing unit that performs a first process using the video data acquired by the first communication unit; a second communication unit that transmits the video data subjected to the first processing by the processing unit to the data processing device via a second communication path. The relay system according to claim 1, wherein the data processing device executes a second process based on the video data that has been subjected to the first process.

Images