Relay device, relay method, relay program, and relay system

Abstract

To provide a relay device, a relay method, a relay program, and a relay system that improve the efficiency of communications.SOLUTION: A first path selection unit 14 selects a first communication path from among multiple communication paths between image capture devices on the basis of information about the image capture device. A first communication unit 11 acquires video data from an image capture device 20 using the first communication path selected by the first path selection unit 14. An AI processing unit 12 executes first processing using the video data acquired by the first communication unit 11. The second communication unit transmits the video data subjected to the first processing by the AI processing unit 12 to a video analysis server 3 using a second communication path.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a relay device, a relay method, a relay program, and a relay system. [Background technology]

[0002] In recent years, there has been an increase in the use of AI (Artificial Intelligence) cameras, which combine imaging devices such as cameras for capturing video with computer systems for analyzing the captured video data. AI cameras are equipped with a GPU that performs high-speed computational processing such as deep learning-based AI, and a wireless or wired communication module. By using such AI cameras, processing such as feature detection and annotation from video data can be performed on the AI ​​camera side, which is expected to improve work efficiency.

[0003] One configuration of AI cameras is one in which video analysis processing is performed by a computer at the edge of the network. AI cameras that adopt this configuration include those equipped with an EAB (Edge Artificial-Intelligence Box), which processes video data using AI and then transmits it via the network to a server or other device running an application.

[0004] In such AI cameras, the EAB acquires video data from a camera connected to the user's network environment, and the results of AI processing within the EAB are sent via the Internet to a server, etc. However, since each user's camera and network environment are different, it is desirable to perform communication between the EAB and the camera in the optimal network environment based on information about the camera and the video being generated.

[0005] As a video data distribution technology, a technology has been proposed in which a camera generates low-quality data from the same video signal and records it on a management server, and records high-quality data on a recording device of the camera, and switches between the low-quality data and the high-quality data and distributes them in response to user operation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-58994 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, image quality (e.g., resolution) and communication environments vary from one imaging device to another, and selecting an appropriate network environment for each imaging device requires a significant workload when switching communication paths in response to user instructions, making it difficult to achieve. Furthermore, if a communication path is selected that can handle even high-quality data, network resources will be wasted by imaging devices that transmit low-quality data. Conversely, if a communication path that supports low-quality data is selected in order to maintain a network environment, the transmission load for imaging devices that transmit high-quality data may become too high, making communication itself difficult. Thus, conventional relay technologies have the potential for wasting network resources and increasing network load, making it difficult to improve communication efficiency.

[0008] The disclosed technology has been made in view of the above, and aims to provide a relay device, a relay method, a relay program, and a relay system that improve the efficiency of communications. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the object, the relay device of the present invention has the following units. A first route selection unit selects a first communication route from among multiple communication routes between the relay device and the image capture device based on information about the image capture device. A first communication unit acquires video data from the image capture device using the first communication route selected by the first route selection unit. A processing unit executes first processing using the video data acquired by the first communication unit. A second communication unit transmits the video data that has been subjected to the first processing by the processing unit to a data processing device using the second communication route. [Effects of the Invention]

[0010] According to the present invention, communication can be made more efficient. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a system configuration diagram of a relay system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of connection between the EAB and the imaging device via a customer LAN. [Figure 3] FIG. 3 is a diagram showing an example of a direct connection of the EAB with an imaging device. [Figure 4] FIG. 4 is a block diagram of an EAB according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of communication path selection. [Figure 6] FIG. 6 is a diagram showing an example of communication path selection accompanied by a change in resolution setting. [Figure 7] FIG. 7 is a diagram showing communication settings of the EAB in response to connection with the video analysis server. [Figure 8] FIG. 8 is a flowchart of a process of determining a communication path by the EAB according to the embodiment. [Figure 9] FIG. 9 is a diagram showing the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of a relay device, a relay method, a relay program, and a relay system disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the relay device, the relay method, the relay program, and the relay system disclosed in the present application are not limited to the following embodiments.

[0013] (Embodiment) (Introduction) The system acquires video from cameras connected to the client's network environment, analyzes the data within the device, and transmits the results via the Internet. The client's network environment is already established and operational, and there were concerns about the impact of adding new video analysis data. Therefore, a relay device was used to simultaneously transmit multiple data using different communication types. However, because the resolution of each camera varies, it is difficult to automatically optimize the communication between the relay device and the camera in the optimal network environment.

[0014] Therefore, the relay device according to the embodiment prioritizes and classifies communication routes in consideration of the amount of data to be communicated and information obtainable from the communication destination, such as the image capture device, and determines the communication route to be used without requiring user operation. Details of the relay device according to the embodiment will be described below.

[0015] (System Configuration) 1 is a system configuration diagram of a relay system according to an embodiment. The relay system 100 includes an EAB 1 which is a relay device, a customer LAN (Local Area Network) 2, a video analysis server 3 on the cloud, and a terminal device 4.

[0016] The customer LAN 2 is provided with IP (Internet Protocol) cameras 21 to 23, which are image capturing devices, a customer terminal device 201, a hub 202, etc. The customer LAN 2 is also connected to the EAB 1.

[0017] The IP cameras 21-23 are connected to the EAB1 via the customer LAN 2. The IP cameras 21-23 are connected to the EAB1 via multiple communication paths. The IP cameras 21-23 may be connected to the EAB1 via a hub 202. For example, the IP cameras 21-23 can each communicate with the EAB1 using a wired connection such as Ethernet (registered trademark), a wireless connection such as Wi-Fi, or a wireless connection such as private 5G. However, these communication paths are merely examples, and other communication paths may also be used as long as they comply with standards that allow data transfer.

[0018] The specifications for imaging and data transfer are predetermined for each of the IP cameras 21 to 23. For example, the resolution, image size, frame rate (fps: Frames Per Second), and video data codec that can be used are predetermined for each of the IP cameras 21 to 23. Here, each of the IP cameras 21 to 23 may be able to use multiple types for each setting. For example, the IP camera 21 may be able to use multiple resolutions.

[0019] The settings used for image capture and data transfer are specified in advance by an administrator for the IP cameras 21 to 23. The IP cameras 21 to 23 store image capture device information that registers the manufacturer name, model name, resolution, image size, frame rate, codec, etc. that can be used by each image capture device 20.

[0020] Each of the IP cameras 21 to 23 captures video according to the specified settings. When the IP cameras 21 to 23 receive an instruction from the EAB 1 to change the resolution setting, they change the setting to the specified resolution and capture video. The IP cameras 21 to 23 then transmit the video data of the captured video to the EAB 1 using the communication path specified by the EAB 1.

[0021] Although the IP cameras 21 to 23 have been described as an example of the imaging devices, the imaging devices may be other devices that can capture video images.

[0022] The customer terminal device 201 can be connected to the IP cameras 21 to 23 via a hub 202 or the like. The administrator of the IP cameras 21 to 23 can use the customer terminal device 201 to configure the settings of the IP cameras 21 to 23.

[0023] The EAB 1 is connected to IP cameras 21 to 23 via a customer LAN 2. Fig. 2 is a diagram showing an example of connection of the EAB to an imaging device via the customer LAN 2. For example, as shown in Fig. 2, the EAB 1 has a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103.

[0024] The EAB 1 can communicate with the IP cameras 21 to 23 via the customer LAN 2 using one or more of the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. For example, in FIG. 2, the EAB 1 is connected to the customer LAN 2 using the Wi-Fi communication interface 103. In this case, the EAB 1 can communicate with the IP cameras 21 to 23 using a communication path via Wi-Fi communication.

[0025] 2 shows a state in which the EAB 1 is connected via one path, the EAB 1 may be connected to the customer LAN 2 via multiple interfaces, among the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. In this case, the EAB 1 can select a communication path from the connected paths to communicate with the IP cameras 21 to 23.

[0026] Fig. 3 is a diagram showing an example of a direct connection between an EAB and an imaging device. For example, as shown in Fig. 3, the EAB 1 may be directly connected to an IP camera 21 via a network. In Fig. 3, the EAB 1 is connected to the IP camera 21 using a wired communication interface 102. However, the EAB 1 may be connected to the IP camera 21 via a communication path using another interface depending on the network function of the IP camera 21, or may be connected via multiple communication paths.

[0027] Returning to FIG. 1, the explanation continues. The EAB 1 identifies the type of communication path that is optimal for transmitting and receiving video data for each of the IP cameras 21-23 from among multiple communication paths connecting the IP cameras 21-23, according to the resolution of the video generated by each of the IP cameras 21-23. The EAB 1 then notifies each of the IP cameras 21-23 of the identified type of communication path as the communication path to use for transmitting the video data. If the identified type of communication path is unavailable, the EAB 1 determines an available communication path that corresponds to a lower resolution as the communication path to use. The EAB 1 then instructs the IP cameras 21-23 to change their settings to transmit video data via the communication path that has been determined to be used, and notifies them of the communication path to use for transmitting the video data.

[0028] Furthermore, the EAB1 executes AI processing using a designated AI application for each piece of video data transmitted from the IP cameras 21 to 23. Here, the EAB1 can use the same AI application or different applications to process each piece of data.

[0029] Furthermore, the EAB1 determines, according to the designated priority, a communication path to be used for transmitting the video data and the results of the AI ​​processing from among multiple communication paths connecting the EAB1 to the video analysis server 3 on the cloud. The EAB1 then transmits the results of the AI ​​processing and the video data to the video analysis server 3 using the determined communication path. The EAB1 will be described in detail later.

[0030] The video analysis server 3 is a device deployed on the cloud. The video analysis server 3 is connected to the EAB1 via the Internet. The video analysis server 3 receives the results of AI processing and the video data from the EAB1. The video analysis server 3 then performs processing to add value to the video data, for example, by visualizing the results of the AI ​​processing. The video analysis server 3 then transmits the processed video data to the terminal device 4. This video analysis server 3 is an example of a "data processing device," and the processing by the video analysis server 3 using the video data and the results of the AI ​​processing is an example of a "second process."

[0031] The terminal device 4 is a device that performs various controls using the images captured by the IP cameras 21 to 23. For example, the terminal device 4 uses the images captured by the IP cameras 21 to 23 to execute control processes for realizing smart retail, smart buildings, smart cities, smart factories, or the like.

[0032] (EAB configuration) Fig. 4 is a block diagram of an EAB according to an embodiment. Next, with reference to Fig. 4, the EAB 1 according to this embodiment will be described in detail. Here, devices for generating images, such as IP cameras 21 to 23, will not be distinguished from one another and will be described as an image capturing device 20. As shown in Fig. 2, the EAB 1 has a first communication unit 11, an AI processing unit 12, a second communication unit 13, a first path selection unit 14, and a second path selection unit 15.

[0033] The first communication unit 11 is connected to one or more image capture devices 20 via multiple communication paths. The multiple communication paths between the first communication unit 11 and each image capture device 20 are an example of the "first multiple communication paths." For example, the first communication unit 11 has a communication path for a wired connection via Ethernet communication, a communication path for a wireless connection via Wi-Fi communication, and a communication path for a wireless connection via private 5G communication. However, the communication paths available between the first communication unit 11 and each image capture device 20 do not have to be the same.

[0034] The first communication unit 11 receives camera device information including information on the image quality of the video data generated by the camera device 20. For example, when the EAB1 is connected to the customer LAN 2, the first communication unit 11 acquires camera device information in which information on settings such as resolution is registered from each camera device 20 connected to the customer LAN 2 using one of the available communication paths.

[0035] Then, the first communication unit 11 outputs the image capturing device information to the first path selection unit 14. After that, the first communication unit 11 receives information on the communication path to be used by the image capturing device 20 in transmitting the video data from the first path selection unit 14 and transmits the information to the image capturing device 20.

[0036] Furthermore, when the first communication unit 11 receives an input of an instruction to change the resolution setting from the first path selection unit 14, it transmits the instruction to change the resolution setting to the targeted image capture device 20, causing the image capture device 20 to change the resolution setting. Then, the first communication unit 11 receives information on a communication path corresponding to the changed resolution from the first path selection unit 14 and transmits it to the image capture device 20 as information on the communication path to be used by the image capture device 20 in transmitting video data.

[0037] Thereafter, the first communication unit 11 receives the video data from the image capturing device 20 via the communication path designated by the first path selection unit 14. Then, the first communication unit 11 outputs the received video data to the AI ​​processing unit 12.

[0038] The first path selection unit 14 receives input of camera device information for each camera device 20 from the first communication unit 11. The camera device information includes information indicating the image quality such as the resolution, image size, frame rate, and codec that can be used by each camera device 20, in addition to the manufacturer name and model name. The first path selection unit 14 performs the following path selection process for each camera device 20. Since the path selection process for each camera device 20 is performed in the same way, the path selection process will be described below using one camera device 20 as an example.

[0039] The first path selection unit 14 acquires the resolution of the image capture device 20 from the image capture device information. Here, the first path selection unit 14 holds information on the type of communication path to be used for each resolution. For example, the first path selection unit 14 divides the resolution into three categories, high, medium, and low, according to the height, and holds information on the type of communication path to be used for each category. Specifically, the first path selection unit 14 holds information such as using a communication path via wired communication for resolutions in the high category, a communication path via private 5G communication for resolutions in the medium category, and a communication path via Wi-Fi communication for resolutions in the low category. However, this division of resolution is not limited to three levels, and may be two levels or four or more levels, and the type of communication path corresponding to each resolution may be determined.

[0040] The first path selection unit 14 identifies the type of communication path according to the resolution of the camera device 20 acquired from the camera device information. Next, the first path selection unit 14 determines whether communication with the camera device 20 is possible via the identified type of communication path. For example, if the identified type of communication path does not exist between the camera device 20 and the first path selection unit 14 or if a communication path of the identified type exists but is difficult to use due to a failure, the first path selection unit 14 determines that communication with the camera device 20 is not possible via the identified type of communication path.

[0041] If communication is possible, the identified type of communication path is determined as the communication path to be used for transmitting and receiving video data from the image capture device 20. Thereafter, the first path selection unit 14 notifies the image capture device 20 via the first communication unit 11 of information on the communication path to be used by the image capture device 20 for transmitting the video data.

[0042] 5 is a diagram showing an example of communication path selection. For example, a case will be described in which the EAB 1 is connected to each of the IP cameras 21 to 23 via a communication path 211 for wired communication, a communication path 212 for Wi-Fi communication, and a communication path 213 for private 5G communication. Also, a case will be described in which the first path selection unit 14 divides resolution into three sections, high, medium, and low, and uses the communication paths 211, 213, and 212 in order of highest resolution.

[0043] The first path selection unit 14 confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. Then, the first path selection unit 14 specifies a communication path 211 via wired communication corresponding to the high category as an appropriate communication path for transmitting and receiving video data to and from the IP camera 21. Furthermore, the first path selection unit 14 confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. Then, the first path selection unit 14 specifies a communication path 213 via private 5G communication corresponding to the medium category as an appropriate communication path for transmitting and receiving video data to and from the IP camera 22. Furthermore, the first path selection unit 14 confirms from the imaging device information of the IP camera 23 that the IP camera 23 uses a resolution included in the low category. Then, the first path selection unit 14 specifies a communication path 212 via Wi-Fi communication corresponding to the low category as an appropriate communication path for transmitting and receiving video data to and from the IP camera 23. Since communication with the IP cameras 21 to 23 is possible via any of the communication paths, the first path selection unit 14 determines each of the communication paths as the communication path to be used. As a result, the first communication unit 11 communicates with the IP cameras 21 to 23 using the communication path corresponding to each resolution, as shown in FIG.

[0044] 4, the explanation will be continued. On the other hand, when communication is not possible via the specified type of communication path because the specified type of communication path is not connected to the image capture device 20 or is unavailable due to a failure, the first path selection unit 14 determines whether communication with the image capture device 20 is possible via a communication path that supports a lower resolution. Specifically, the first path selection unit 14 determines whether communication with the image capture device 20 is possible via a communication path that supports a lower resolution, and whether the image capture device 20 supports that resolution.

[0045] If a communication path corresponding to a lower resolution is available, the first path selection unit 14 determines, as the communication path to be used, a communication path having the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the available communication paths. This communication path is a communication path corresponding to the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the communication paths that the EAB1 can use between the camera device 20 and the EAB1. Next, the first path selection unit 14 transmits, to the camera device 20 via the first communication unit 11, an instruction to change the setting to the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the determined communication path. Thereafter, the first path selection unit 14 notifies the camera device 20 via the first communication unit 11 of information about the communication path to be used by the camera device 20 to transmit video data.

[0046] 6 is a diagram showing an example of communication path selection involving a change in resolution setting. For example, a case will be described in which the EAB 1 is connected to each of the IP cameras 21 to 23 via a communication path 212 based on Wi-Fi communication, and the communication path 211 based on wired communication and the communication path 213 based on private 5G communication are unavailable due to a failure or the like. Again, the first path selection unit 14 divides the resolution into three categories: high, medium, and low, and uses the communication paths 211, 213, and 212 in descending order of resolution.

[0047] In this case, too, the first path selection unit 14 confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. Then, the first path selection unit 14 determines whether the communication path 211 corresponding to the high category is available. In this case, the first path selection unit 14 determines that the communication path 211 is unavailable and that the communication path 212 using Wi-Fi communication is available. Then, the first path selection unit 14 decides to use the communication path 212 using Wi-Fi, which corresponds to the low category, for sending and receiving video data to and from the IP camera 21. Thereafter, the first path selection unit 14 instructs the IP camera 21 to change the setting to a resolution that is available for the IP camera 21 and is as high as possible, from among the resolutions available on the communication path 212. Furthermore, the first path selection unit 14 confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. Then, the first path selection unit 14 determines whether the communication path 213 corresponding to the medium category is available. In this case, the first path selection unit 14 determines that the communication path 213 cannot be used and that the communication path 212 using Wi-Fi communication is usable. The first path selection unit 14 then determines to use the communication path 212 using Wi-Fi communication corresponding to the low category for transmitting and receiving video data to and from the IP camera 22. The first path selection unit 14 then instructs the IP camera 22 to change the setting to a resolution that is usable by the IP camera 22 and is as high as possible, among the resolutions usable on the communication path 212. The first path selection unit 14 also confirms from the imaging device information of the IP camera 23 that the IP camera 23 uses a resolution included in the low category. In this case, since the communication path 211 using Wi-Fi corresponding to the low category is usable, the first path selection unit 14 determines to use the communication path 211 for transmitting and receiving video data to and from the IP camera 23. The first path selection unit 14 then communicates with all of the IP cameras 21 to 23 using the communication path 212, as shown in FIG. 5 .

[0048] For example, the communication path 211 corresponding to the resolution of the IP camera 21 is an example of a "first communication path," and the inability to communicate between the IP camera 21 and the EAB1 via the first communication path is an example of a "case where it is difficult to select the first communication path." Furthermore, the communication path 212 selected by the first path selection unit 14, which is other than the communication path 211 suitable for the resolution of the IP camera 21, is an example of a "third communication path." That is, when it is difficult to select the communication path 211, which is the first communication path based on the information about the IP camera 21, the first path selection unit 14 selects the communication path 212, which is the third communication path other than the communication path 211 suitable for the resolution of the IP camera 21, from among the communication paths between the image capture device 20, and changes the settings of the IP camera 21 so that the video data is transmitted via the third communication path.

[0049] In this embodiment, the resolution classifications are associated one-to-one with different types of communication paths 211 to 213, and the first path selection unit 14 selects a communication path corresponding to the resolution. Furthermore, in order to improve the image quality of the video data, the first path selection unit 14 changes the settings of the image capture device 20 so that the highest possible resolution can be used among the available communication paths. However, the methods for selecting a communication path for a resolution and changing the resolution settings of the image capture device 20 are not limited to this.

[0050] For example, the resolution categories can be associated with the communication paths 211 to 213 as follows: The communication path 211 using wired communication can transmit video data in all resolution categories, high, medium, and low. The communication path 212 using private 5G communication can transmit video data in two resolution categories, medium and low. In contrast, the communication path 213 using Wi-Fi communication can transmit only video data in the low resolution category. In this case, the first path selection unit 14 may determine the communication path that the image capture device 20 will use to transmit video data as follows.

[0051] For example, if the resolution of the image capturing device 20 is in the low category, all of the communication paths 211 to 213 are available, and the network resources of the wired communication path 211 are greater than those of the communication paths 212 and 213, the first path selection unit 14 may use the wired communication path 211 for transmitting video data from the image capturing device 20 in order to make effective use of the network resources. In this case, the selection of the communication path 211 based on the resolution and network resources of the image capturing device 20 is an example of "selecting a first communication path based on information about the image capturing device."

[0052] In this case, if multiple image capture devices 20 with low resolution are connected to the EAB1 and all of the communication paths 211 to 213 are available, connecting all of the image capture devices 20 to the wired communication path 211 may place a load on the network. Therefore, when the communication speed of the communication path 211 falls below a threshold, the first path selection unit 14 may change the communication path 211 to use the private 5G communication path 212 for transmitting video data from some of the image capture devices 20 so that the communication speed falls within the threshold. Similarly, when the communication speed of the communication path 212 falls below a threshold, the first path selection unit 14 may change the communication path 212 to use the Wi-Fi communication path 213 for transmitting video data from some of the image capture devices 20 so that the communication speed falls within the threshold.

[0053] The same applies when the resolution of the image capture device 20 is included in the high category, and when the communication speed of the communication path 211 falls below a threshold, the first path selection unit 14 may change the communication path 212 to use private 5G communication for transmitting video data from some image capture devices 20 so that the communication speed falls within the threshold. In this case, the first path selection unit 14 instructs the target image capture device 20 to change the resolution setting to the medium category in order to use the communication path 212 using private 5G communication.

[0054] The case where the communication speed falls below the threshold is an example of a case where it is difficult to select the first communication path. The communication path 212 selected instead of the communication path 211 is an example of a third communication path.

[0055] Furthermore, in this embodiment, the first path selection unit 14 determines the communication path according to the resolution, which defines the data volume of the video data generated by the image capture device 20. However, the criteria for defining the data volume for determining the communication path are not limited to this, and other information can be used as long as it is information that can determine the amount of data to be transmitted. For example, the data volume can also be defined by other information that indicates the image quality of the video, such as image size, frame rate (FPS), bit rate, or codec. Therefore, the first path selection unit 14 may acquire such information that indicates the image quality of the video from the image capture device information and determine the communication path to be used.

[0056] Here, the first route selection unit 14 may identify this information from the manufacturer name and model name of the photographing device 20 registered in the photographing device information. Alternatively, the first route selection unit 14 may combine some of this information and use it to determine the communication route.

[0057] Furthermore, the first path selection unit 14 may acquire the radio wave intensity of wireless communication with the image capture device 20 and use it to select a communication path, for example, not to use wireless communication with radio wave intensity below a reference value. Additionally, the first path selection unit 14 may use the type of AI application compatible with the image capture device 20, which is used for AI processing in the AI ​​processing unit 12 for each piece of video data, to select a communication path.

[0058] For example, when receiving video data to be processed by an AI application that can tolerate a small drop in data, the first path selection unit 14 can select a communication path that can transfer a small amount of data. In other words, the first path selection unit 14 can select a communication path based on the video quality required for the AI ​​application to maintain a minimum level of performance.

[0059] In this way, the first path selection unit 14 can determine a communication path to be used for transmitting video data to and from the image capture device 20, based on various pieces of information about the image capture device 20, such as information indicating the image quality, radio wave intensity, and the type of AI application. Here, the first path selection unit 14 selected a communication path corresponding to the information acquired based on a predetermined correspondence, but the present invention is not limited to this and the communication path may be determined using, for example, AI. The communication path selected by the first path selection unit 14 to be used for transmitting video data between the first communication unit 11 and the image capture device 20 is an example of a "first communication path."

[0060] Returning to Figure 4, the explanation will continue. The AI ​​processing unit 12 receives input of video data transmitted from each of the multiple image capture devices 20 from the first communication unit 11. The AI ​​processing unit 12 has multiple AI applications. For example, one AI application may be an application for detecting human faces, and another AI application may be an application for detecting automobiles, and each AI application may execute a different AI process.

[0061] The AI ​​processing unit 12 performs AI processing on each piece of acquired video data using a designated AI application. The AI ​​processing unit 12 may retain information predetermined for each image capture device 20 as the designation of the AI ​​application to be used, or may acquire the designation of the application to be used from information added to the video data. The AI ​​processing unit 12 then outputs the result of the AI ​​processing together with the video data to the second communication unit 13. This AI processing unit 12 is an example of a "processing unit," and the AI ​​processing of the video data by the AI ​​processing unit 12 is an example of a "first processing."

[0062] The second path selection unit 15 receives, from an input unit (not shown), an input specifying the priority of each of the multiple communication paths connecting the second communication unit 13 and the video analysis server 3. The multiple communication paths connecting the second communication unit 13 and the video analysis server 3 are an example of "second multiple communication paths." The second path selection unit 15 then determines the communication path with the highest priority among the available communication paths as the communication path to be used for communication with the video analysis server 3. The communication path selected by the second path selection unit 15 to be used for transmitting video data and AI processing results between the second communication unit 13 and the video analysis server 3 is an example of a "second communication path."

[0063] For example, as shown in Fig. 5, the EAB 1 has a communication path 301 based on a wired connection using Ethernet, a communication path 302 based on 5G communication, a communication path 303 based on LTE (Long Term Evolution) communication, and a communication path 304 based on Wi-Fi. The second path selection unit 15 receives designation of the communication paths in the order of decreasing priority, for example, communication path 301, communication path 302, communication path 303, and communication path 304. In this case, the second path selection unit 15 determines the communication path 301 based on a wired connection, which has the highest priority among the available communication paths, as the communication path to be used for communication with the video analysis server 3. As a result, the EAB 1 communicates with the video analysis server 3 using the communication path 301 based on a wired connection.

[0064] Returning to Figure 4, the explanation will continue. The second path selection unit 15 instructs the second communication unit 13 to communicate with the video analysis server 3 using the communication path that has been decided to be used for communication. Then, the second path selection unit 15 sets the communication path that has been decided to be used for communication with the video analysis server 3 to ON in the communication settings of the second communication unit 13, and sets other communication paths to OFF. However, when the first communication unit 11 and the second communication unit 13 communicate using the same communication interface, the second path selection unit 15 maintains the ON setting for the communication path used by the first communication unit 11.

[0065] 7 is a diagram showing communication settings of the EAB 1 in response to connection with the video analysis server. Here, the EAB 1 is described as having, as communication interfaces, a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103. It is also assumed that the wired communication interface 102 is used for communication between the EAB 1 and the customer LAN 2.

[0066] Connection state 111 indicates a state in which the EAB 1 and the video analysis server 3 communicate via mobile data communication. In this case, the second path selection unit 15 turns on the setting for communication via the mobile data communication interface 101, leaves the setting for communication via the wired communication interface 102 on, and turns off the setting for communication via the Wi-Fi communication interface 103.

[0067] The connection state 112 indicates a state in which the EAB 1 and the video analysis server 3 communicate via Wi-Fi. In this case, the second path selection unit 15 turns off the setting for communication via the mobile data communication interface 101, leaves the setting for communication via the wired communication interface 102 on, and turns on the setting for communication via the Wi-Fi communication interface 103.

[0068] The connection state 112 indicates a state in which the EAB 1 and the video analysis server 3 communicate via wired communication. In this case, the second path selection unit 15 leaves the setting for communication via the wired communication interface 102 on, and turns off the settings for communication via the wired communication interface 102 and the Wi-Fi communication interface 103.

[0069] Continuing the explanation, returning to Figure 4, the second communication unit 13 is connected to the video analysis server 3 via one or more communication paths. For example, the second communication unit 13 may have a communication path for a wired connection using Ethernet communication, a communication path for a wireless connection using 5G communication, a communication path for wireless communication using LTE communication, and a communication path for a wireless connection using Wi-Fi.

[0070] The second communication unit 13 receives from the second path selection unit 15 a specification of a communication path to be used for communication with the video analysis server 3. Then, the second communication unit 13 transmits the video data and the results of the AI ​​processing to the video analysis server 3 using the communication path specified by the second path selection unit 15. At this time, the second communication unit 13 can transmit each piece of video data and the results of the AI ​​processing to the video analysis server 3 together using the same communication path.

[0071] (Relay processing) 8 is a flowchart of a process of determining a communication path by the EAB 1 according to the embodiment. Next, the flow of the process of determining a communication path by the EAB 1 according to the embodiment will be described with reference to FIG.

[0072] The first route selection unit 14 acquires the photographing device information of each of the photographing devices 20 connected to the customer LAN 2 (step S1).

[0073] Next, the first route selection unit 14 selects one image capturing device 20 (step S2).

[0074] Next, the first route selection unit 14 identifies a communication route suitable for transmitting and receiving video data to and from the selected camera device 20 based on the acquired camera device information (step S3).

[0075] Next, the first route selection unit 14 determines whether the identified communication route exists as a communication route connecting with the image capture device 20 and is usable as a communication route to be used for transmitting and receiving video data (step S4).

[0076] If the identified communication path is available for use (step S4: Yes), the first path selection unit 14 determines the identified communication path as the communication path to be used (step S5). After that, the communication path determination process proceeds to step S8.

[0077] On the other hand, if it is difficult to use as the communication path to be used (step S4: No), the first path selection unit 14 extracts the highest possible resolution that can be used by the selected image capture device 20 from among the resolutions corresponding to the available communication paths between the image capture device 20. Then, the first path selection unit 14 determines the communication path corresponding to the extracted resolution from among the available communication paths as the communication path to be used (step S6).

[0078] Then, the first route selection unit 14 notifies the image capture device 20 via the first communication unit 11 of an instruction to change the settings to the resolution corresponding to the determined communication route, and changes the settings of the image capture device 20 (step S7). Thereafter, the communication route determination process proceeds to step S8.

[0079] Thereafter, the first route selection unit 14 notifies the image capturing device 20 of the communication route that has been determined to be used (step S8).

[0080] Next, the first route selection unit 14 determines whether or not the determination of the communication routes has been completed for all the camera devices 20 connected to the customer LAN 2 (step S9). If there are any camera devices 20 for which the communication routes have not been determined (step S9: No), the communication route determination process returns to step S2.

[0081] On the other hand, if the determination of the communication paths for all the image capture devices 20 has been completed (step S9: Yes), the first path selection unit 14 completes the setting of the communication paths with the image capture devices 20. Next, the second path selection unit 15 acquires from the input unit the priority of the communication paths to be used for sending and receiving the video data and the results of the AI ​​processing with the video analysis server 3 (step S10).

[0082] Thereafter, the second route selection unit 15 determines, in accordance with the priority, a communication route to be used with the video analysis server 3. Then, the second route selection unit 15 performs communication settings for the second communication unit 13 so that video data is transmitted and received with the video analysis server 3 using the determined communication route (step S11).

[0083] The first communication unit 11 receives the video data from each of the image capturing devices 20 via the determined communication path between the first communication unit 11 and each of the image capturing devices 20 (step S12).

[0084] The AI ​​processing unit 12 performs AI processing on the video data received by the first communication unit 11 (step S13).

[0085] The second communication unit 13 transmits the video data received from each image capture device 20 and the results of each AI processing to the video analysis server 3 via the determined communication path between the second communication unit 13 and the video analysis server 3 (step S13).

[0086] (effect) As described above, the EAB1 according to this embodiment determines the type of communication path to use based on the amount of data determined by the image quality, such as the resolution, of the video data generated by the image capture device 20. This makes it possible to select the optimal communication path for each image capture device 20, automatically suppressing the waste of network resources and the increase in network load, and improving communication efficiency.

[0087] Furthermore, when the optimal communication path obtained from the amount of video data to be transmitted cannot be used, the EAB1 according to this embodiment changes the settings of the image capture device 20 to reduce the amount of video data, and selects a suitable communication path from among the communication paths with reduced data amounts as the next best communication path. This makes it possible to minimize waste of network resources and increases in network load, and improve communication efficiency, even when the communication path considered optimal cannot be used.

[0088] That is, EAB1, which is a relay device according to the embodiment, has a first route selection unit 14 that selects a first communication route from among a first plurality of communication routes between the camera device 20 based on information about the camera device 20, a first communication unit 11 that acquires video data from the camera device 20 using the first communication route selected by the first route selection unit 14, an AI processing unit 12 that performs first processing using the video data acquired by the first communication unit 11, and a second communication unit 13 that transmits the video data that has been subjected to the first processing by the AI ​​processing unit 12 to the data processing device using a second communication route.

[0089] This allows the EAB1 to automatically select the optimal communication path to receive video data from the imaging device 20, automatically reducing the waste of network resources and the increase in network load, thereby improving communication efficiency.

[0090] In the EAB1, which is a relay device according to the embodiment, the first route selection unit 14 selects the first communication route based on information that specifies the amount of video data included in the information of the image capturing device 20.

[0091] This allows the EAB 1 to select an appropriate communication path depending on the amount of data, thereby ensuring efficient communication.

[0092] In the EAB1, which is a relay device according to the embodiment, the first path selection unit 14 selects the first communication path based on information indicating image quality as information defining the data amount of video data.

[0093] This allows the EAB1 to select an appropriate communication path according to the amount of data based on the image quality, thereby ensuring more efficient communication.

[0094] The EAB1, which is a relay device in this embodiment, further includes a second path selection unit 15 that selects a second communication path based on the priority of each of the second multiple communication paths between the EAB1 and the video analysis server 3, which is a data processing device, and the second communication unit 13 transmits the video data using the second communication path selected by the second path selection unit 15.

[0095] This allows the EAB1 to automatically select the optimal communication route and send video data and AI processing results to the video analysis server 3, automatically reducing the waste of network resources and the increase in network load, thereby improving communication efficiency.

[0096] (program) It is also possible to create a program written in a computer-executable language that executes the processes executed by the EAB1 described in the above embodiment. In this case, the same effects as those of the above embodiment can be achieved by having a computer execute the program. Furthermore, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer to achieve the same processes as those of the above embodiment.

[0097] Fig. 9 is a hardware configuration diagram of a computer. The EAB 1 is realized, for example, by a computer 90 shown in Fig. 9. The computer 90 has, for example, a processor 91, a memory 92, a hard disk 93, and a communication interface 94, as shown in Fig. 9.

[0098] The processor 91 operates based on programs stored on the hard disk 93 and controls each component. The memory 92 stores, for example, a boot program executed by the processor 91 when the computer 90 starts up. The hard disk 93 stores the programs executed by the processor 91 and data used by the programs.

[0099] The communication interface 94 receives data from other devices via a communication network NW (Network) and sends it to the processor 91, and transmits data generated by the processor 91 to other devices via the communication network NW. The communication interface 94 realizes the functions of the first communication unit 11 and the second communication unit 13 illustrated in FIG.

[0100] The processor 91 reads out various programs stored in the hard disk 93, expands them into the memory 92, and executes them, thereby realizing the functions of the AI ​​processing unit 12, the first path selection unit 14, and the second path selection unit 15 illustrated in Fig. 4. The processor 91 reads out and executes the programs from the hard disk 93, but as another example, the processor 91 may obtain these programs from other devices via the communication network NW.

[0101] (others) Although various embodiments have been described in detail herein with reference to the drawings, these embodiments are merely examples and are not intended to limit the present invention. The features described herein can be realized in various ways, including various modifications and improvements based on the knowledge of those skilled in the art.

[0102] Furthermore, the above-mentioned "module (-er suffix, -or suffix)" can be read as a unit, means, circuit, step, etc. For example, a communication module, a processing module, and a selecting module can be read as a communication unit, a control unit, and a storage unit, respectively. [Explanation of symbols]

[0103] 1 EAB 2 Customer LAN 3. Video analysis server 4 Terminal Devices 11 First Communications Department 12 AI processing section 13 Second Communications Department 14 First route selection unit 15 Second Route Selection Unit 20 Imaging equipment 21~23 IP Camera 100 Relay System 201 Customer terminal equipment 202 Hub

Claims

1. A first path selection unit that selects, for each of a plurality of photographing devices, a first communication path from among a plurality of communication paths between the photographing device based on information about the photographing device, and when there are a plurality of photographing devices that have selected the same specific communication path as the first communication path and the communication speed of the specific communication path falls below a threshold, reselects, for at least one or more of the photographing devices for which the specific communication path has been selected, a communication path having a lower transfer rate than the specific communication path as the first communication path; a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit; a processing unit that executes a first process using the video data acquired by the first communication unit; a second communication unit that transmits the video data that has been subjected to the first processing by the processing unit to a data processing device via a second communication path; A relay device comprising:

2. The relay device according to claim 1 , wherein the first route selection unit selects the first communication route based on information specifying the amount of video data included in the information of the imaging device.

3. 3. The relay device according to claim 2, wherein the first path selection unit selects the first communication path based on information representing image quality as information defining the data amount of the video data.

4. a second path selection unit that selects the second communication path based on the priority of each of a plurality of communication paths between the data processing device and the second communication path selection unit; The second communication unit transmits the video data using the second communication path selected by the second path selection unit.

2. The relay device according to claim 1, wherein:

5. The relay device according to claim 1 , wherein the first route selection unit transmits to each of the image capturing devices an instruction to change the settings of the information of the image capturing devices used to select each of the first communication routes.

6. when it is difficult to select the first communication path from among the plurality of communication paths based on the information of the image capturing device, the first path selection unit selects a third communication path other than the first communication path from among the plurality of communication paths based on the information of the image capturing device, and changes a setting of the image capturing device so that the image capturing device transmits video data via the third communication path; The first communication unit acquires video data from an image capturing device using the third communication path selected by the first path selection unit.

2. The relay device according to claim 1, wherein:

7. A relay device, a first route selection step of selecting, for each of a plurality of photographing devices, a first communication route from among a plurality of communication routes between the photographing device and the respective photographing devices based on information about the photographing device, and when a plurality of photographing devices have selected the same specific communication route as the first communication route and the communication speed of the specific communication route falls below a threshold, reselecting, as the first communication route, a communication route having a lower transfer rate than the specific communication route for at least one of the photographing devices for which the specific communication route has been selected; a first communication step of acquiring video data from an image capturing device using the first communication path selected by the first path selection step; a processing step of executing a first process using the video data acquired by the first communication step; a second communication step of transmitting the video data subjected to the first processing by the processing step to a data processing device via a second communication path; A relay method comprising:

8. A first route selection step for selecting, for each of a plurality of photographing devices, a first communication route from a plurality of communication routes between the photographing devices based on information about the photographing device, and when there are a plurality of photographing devices which have selected the same specific communication route as the first communication route and the communication speed of the specific communication route falls below a threshold, reselecting, for at least one or more of the photographing devices for which the specific communication route has been selected, a communication route having a lower transfer rate than the specific communication route as the first communication route; a first communication step of acquiring video data from an image capturing device using the first communication path selected in the first path selection step; a processing step of performing a first process using the video data acquired in the first communication step; a second communication step of transmitting the video data subjected to the first processing by the processing step to a data processing device via a second communication path; A relay program that causes a computer to execute the above.

9. A relay system having a plurality of imaging devices, a relay device, and a data processing device, the plurality of image capturing devices are each connected to the relay device via a plurality of communication paths; The relay device a first path selection unit that selects, for each of the plurality of image capture devices, a first communication path from among the plurality of communication paths based on information about the image capture device, and when there are a plurality of image capture devices that have selected the same specific communication path as the first communication path and the communication speed of the specific communication path falls below a threshold, reselects, as the first communication path, a communication path having a lower transfer rate than the specific communication path for at least one of the image capture devices for which the specific communication path has been selected; a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit; a processing unit that performs a first process using the video data acquired by the first communication unit; a second communication unit that transmits the video data that has been subjected to the first processing by the processing unit to the data processing device using a second communication path; The relay system is characterized in that the data processing device executes a second process based on the video data that has been subjected to the first process.

Images