Processing control system, processing control device, and processing control method

Abstract

In order to make it possible to accommodate communication bandwidth fluctuations, this processing control system is provided with: a load prediction means (101) that predicts the processing load of data to be analyzed on a first processing unit (20); a bandwidth prediction means (102) that predicts the communication bandwidth between the first processing unit and a second processing unit (30); and a switching control means (110) that controls whether the first processing unit or the second processing unit analyzes the data to be analyzed, on the basis of the predicted processing load and the predicted communication bandwidth.

Description

Technical Field

[0001] The present invention relates to a processing control system, a processing control apparatus, and a processing control method.

Background Art

[0002] A technology of processing image data acquired by an imaging device such as a camera to analyze, for example, an object such as a person or an object and the motion thereof is in use. Such processing imposes a heavy load, and therefore distributed processing is often employed. For example, in Patent Document 1, the sequence of image frames is divided and distributed to a plurality of processing devices, and the divided parts are combined after super-resolution processing, thereby performing distributed super-resolution processing. In Patent Document 2, the priority of metadata is calculated, and a wireless bandwidth is allocated to a lower-level server that transfers the metadata in accordance with the priority, thereby preventing delay in metadata transfer.

Prior Art Literature

Patent Literature

[0003]

Patent Literature 1

Patent Literature 2

Summary of the Invention

Problem to be Solved by the Invention

[0004] However, when transferring data, the communication bandwidth is not always stable. That is, the communication bandwidth may fluctuate, and the technologies of Patent Documents 1 and 2 cannot cope with fluctuations in the communication bandwidth, and as a result, there is a possibility that significant processing delays and missing processing may occur.

[0005] One aspect of the present invention has been made in view of the above problem, and an example of the object thereof is to provide a processing control system, a processing control apparatus, and a processing control method that can cope with fluctuations in communication bandwidth.

Means for Solving the Problem

[0006] A processing control system according to one aspect of the present invention is a processing control system that controls a first processing unit and a second processing unit that can communicate with the first processing unit, wherein the first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit, the second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, and the processing control system comprises load prediction means for predicting the processing load of the data to be analyzed in the first processing unit, bandwidth prediction means for predicting the communication bandwidth between the first processing unit and the second processing unit, and switching control means for controlling whether the data to be analyzed is performed by the first processing unit or the second processing unit based on the predicted processing load and the predicted communication bandwidth.

[0007] A processing control device according to one aspect of the present invention is a processing control device that controls a first processing unit and a second processing unit that can communicate with the first processing unit, wherein the first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit, the second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, and the processing control device comprises a load prediction unit that predicts the processing load of the data to be analyzed in the first processing unit, a bandwidth prediction unit that predicts the communication bandwidth between the first processing unit and the second processing unit, and a switching control unit that controls whether the data to be analyzed is performed by the first processing unit or the second processing unit based on the predicted processing load and the predicted communication bandwidth.

[0008] A processing control method according to one aspect of the present invention is a processing control method that controls a first processing unit and a second processing unit that can communicate with the first processing unit. methodA processing control method comprising: a load prediction process that predicts the processing load of the data to be analyzed in the first processing unit; a bandwidth prediction process that predicts the communication bandwidth between the first processing unit and the second processing unit; and a switching control process that controls whether the first processing unit or the second processing unit analyzes the data to be analyzed based on the predicted processing load and the predicted communication bandwidth; wherein the first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit; and the second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit. [Effects of the Invention]

[0009] According to one aspect of the present invention, it is possible to control which of the multiple processing units analyzes the data to be analyzed in response to fluctuations in the communication bandwidth. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example configuration of a processing control system according to the first embodiment. [Figure 2] This is a block diagram showing an example configuration of a processing system controlled by a processing control system. [Figure 3] This is a flowchart showing an example of the flow of the processing control method S100 according to the first embodiment. [Figure 4] This is a block diagram showing an example configuration of the processing control device 200 according to the first embodiment. [Figure 5] This is a block diagram showing an example configuration of a processing control system according to the second embodiment. [Figure 6] This is a schematic diagram illustrating an example of the data to be analyzed output from an imaging device. [Figure 7] This is a schematic diagram illustrating an example of the data to be analyzed output from an imaging device. [Figure 8] This graph shows the results of the communication bandwidth prediction method. [Figure 9]It is a block diagram illustrating a configuration example of a processing control system according to a third embodiment. [Figure 10] It is a block diagram illustrating a configuration example of a processing control system according to a fourth embodiment. [Figure 11] It is a block diagram illustrating a configuration example of a processing control system according to a fifth embodiment. [Figure 12] It is a block diagram illustrating a configuration example of a processing control system according to a sixth embodiment. [Figure 13] It is a block diagram illustrating a configuration example of a processing control system according to a seventh embodiment. [Figure 14] It is a block diagram illustrating a configuration example of a processing control system according to an eighth embodiment. [Figure 15] It is a block diagram illustrating a configuration example of a processing control system according to a tenth embodiment. [Figure 16] It is a block diagram illustrating a configuration example of a processing control system according to an eleventh embodiment. [Figure 17] It is a block diagram illustrating a configuration example of a processing control system according to a twelfth embodiment. [Figure 18] It is a block diagram illustrating a configuration example of a computer. MODE FOR CARRYING OUT THE INVENTION

[0011] [First Embodiment] A first embodiment of the present invention will be described in detail with reference to the drawings. The present embodiment is a basic form that serves as a basis for embodiments described later.

[0012] (Configuration of Processing Control System) The configuration of the processing control system according to the present embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram illustrating a configuration example of a processing control system 100 according to the first embodiment. The processing control system 100 includes a load prediction unit 101, a bandwidth prediction unit 102, and a switching control unit 110, and controls the processing system.

[0013] Figure 2 is a block diagram showing an example configuration of a processing system controlled by a processing control system. Processing system 1 has a first processing unit 20 and a second processing unit 30.

[0014] The first processing unit 20 is connected to sensors such as cameras and LiDAR (Light Detection and Ranging), and acquires data to be analyzed from the cameras and sensors. In one example, the data to be analyzed may be video data captured by a camera. The video data only needs to include the object to be analyzed within the field of view of the video. The object to be analyzed may be, for example, workers (people), work equipment (objects) working at a construction site, and the behavior (operation) of the workers and work equipment. Alternatively, the data to be analyzed may be sensing data from a sensor that detected the object to be analyzed.

[0015] The first processing unit 20 and the second processing unit 30 may each be composed of one or more computers. The first processing unit 20 and the second processing unit 30 can communicate with each other via a network NW and share the task of analyzing the data to be analyzed. The network NW may be wireless or wired, and in the case of wireless, it may be a wireless communication system such as Wi-Fi, LTE, 4G, or 5G.

[0016] In one embodiment, the first processing unit 20 may be an edge processing unit, and the second processing unit 30 may be cloud processing. In this specification, "edge" refers to a location where data is collected. The first processing unit 20, which is an edge processing unit, is an information processing device (computer) or group of information processing devices installed at or around the location where the object of analysis exists (e.g., a construction site, a factory, etc.), and acquires data to be analyzed from cameras, sensors, etc., installed at the location where the object of analysis exists. The first processing unit 20 may be integrated with the cameras, sensors, etc. In this specification, "cloud" refers to a location where data is processed and stored. The second processing unit 30, which is a cloud processing unit, may be an information processing device (computer) or group of information processing devices installed in a location capable of providing large computing resources, such as a data center or server farm. The second processing unit 30 may be a processing unit located in a location connected to the first processing unit 20 via a network, and may be computing resources connected to a base station such as 5G (e.g., MEC (Multi-access Edge Computing)), or a server installed in an office at the site (on-premises server), etc.

[0017] The division of the analysis of the data to be analyzed between the first processing unit 20 and the second processing unit 30 can be carried out in various ways. For example, the first processing unit 20 may perform the analysis of the data after acquiring it; the first processing unit 20 may perform preprocessing of the data after acquiring it, and the second processing unit 30 may analyze the preprocessed data; or the first processing unit 20 may perform processing such as compression on the data to be analyzed, and the second processing unit 30 may perform the analysis of the data. For example, the first division of labor for analyzing the data to be analyzed may be selected from among the following, depending on the computing power of the first processing unit 20: a first division of labor in which the first processing unit 20 generates the analysis results of the data to be analyzed; a second division of labor in which the first processing unit 20 calculates the feature quantities of the data to be analyzed, transmits the feature quantities from the first processing unit 20 to the second processing unit 30, and generates the analysis results from the feature quantities; and a third division of labor in which the first processing unit 20 transmits the data to be analyzed to the second processing unit 30, and generates the analysis results from the data to be analyzed. In addition to computing power, the criteria used for selecting the division of labor may also include computing cost, importance of the data to be analyzed, risk level indicated by the data to be analyzed, and compression efficiency and communication quality of each data to be analyzed. By using these division of labor methods appropriately, analysis processing can be performed efficiently according to the situation.

[0018] In one embodiment, the first processing unit 20 analyzes at least a portion of the acquired data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit 30. The data to be analyzed transmitted to the second processing unit 30 at this time is at least a portion of the data to be analyzed that has not been fully processed by the first processing unit 20 (the remaining data to be analyzed). The first processing unit 20 transmits at least a portion of the data to be analyzed (for example, at least a portion of the remaining data to be analyzed) to the second processing unit 30 via the network NW. The second processing unit 30 receives the data to be analyzed transmitted from the first processing unit 20 (for example, at least a portion of the remaining data to be analyzed) and analyzes it.

[0019] Furthermore, the data to be analyzed transmitted from the first processing unit 20 to the second processing unit 30 may be pre-processed in the first processing unit 20. For example, the first processing unit 20 may calculate the features of the data to be analyzed, transmit these features to the second processing unit 30, and the second processing unit 30 may analyze these features. In this specification, the data to be analyzed includes data that has been pre-processed (e.g., features). Also, in this specification, analyzing the data to be analyzed means generating the analysis results of the data to be analyzed, and merely pre-processing the data to be analyzed does not constitute analyzing the data to be analyzed.

[0020] The analysis of the data to be analyzed includes, for example, the detection, identification, tracking, and time-series analysis of the target of analysis (objects, people) in the video. AI may be used for processing this data to be analyzed. Either or both of the first processing unit 20 and the second processing unit 30 may use AI.

[0021] The processing control system 100 (load prediction means 101, bandwidth prediction means 102, switching control means 110) controls the processing system 1, in particular the first processing unit 20 and the second processing unit 30.

[0022] The load prediction means 101 predicts the processing load of the data to be analyzed in the first processing unit 20. The processing load is, for example, the amount of computing resources used in the first processing unit 20 to process the data to be analyzed (including processing for analysis and preprocessing) (the amount of CPU and GPU usage required to process the data to be analyzed per unit time). The load prediction means 101 can predict the future processing load by, for example, monitoring the temporal changes in the processing load of the data to be analyzed in the first processing unit 20 (for example, the number of people to be processed, the size of the people to be processed, the amount of computing resources used, the processing speed, or a combination thereof). Alternatively, the processing load may be predicted based on the processing speed of the data to be analyzed (the amount of data to be analyzed processed per unit time).

[0023] The bandwidth prediction means 102 predicts the communication bandwidth between the first processing unit 20 and the second processing unit 30. The communication bandwidth is, for example, the data transfer rate (amount of data transferred per unit time) that can be transferred between the first processing unit 20 and the second processing unit 30. The bandwidth prediction means 102 can predict the future communication bandwidth by, for example, monitoring the temporal change in the communication bandwidth (e.g., transfer rate) between the first processing unit 20 and the second processing unit 30.

[0024] The switching control means 110 controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the processing load predicted by the load prediction means 101 and the communication bandwidth predicted by the bandwidth prediction means 102.

[0025] The switching control means 110 switches the analysis of the data to be analyzed from the first processing unit 20 to the second processing unit 30 if, for example, the predicted processing load in the first processing unit 20 approaches the limit of the processing speed of the first processing unit 20 during processing in the first processing unit 20. The switching control means 110 also switches the analysis of the data to be analyzed from the second processing unit 30 to the first processing unit 20 if, for example, the required data transfer speed (e.g., transfer speed from a camera, etc.) approaches the lower limit of the predicted transmittable bandwidth (lower limit bandwidth Bmin, described later) during processing in the second processing unit 30. When a portion of the data to be analyzed (the portion of the data to be analyzed) is analyzed in the first processing unit 20, the first processing unit 20 analyzes the portion of the data to be analyzed and does not need to transmit the portion of the data to be analyzed to the second processing unit 30. When the data to be analyzed (the portion of the data to be analyzed) is analyzed in the second processing unit 30, the first processing unit 20 does not analyze the portion of the data to be analyzed, but processes it for suitability and then transmits it to the second processing unit 30. The second processing unit 30 analyzes at least a portion of the data to be analyzed transmitted from the first processing unit 20.

[0026] As described above, the processing control system 100 according to this embodiment controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the predicted processing load and communication bandwidth. Therefore, according to the processing control system 100 according to this embodiment, processing in the first processing unit 20 and the second processing unit 30 can be switched based on the communication bandwidth.

[0027] (Processing control method flow) The flow of the processing control method S100 according to this embodiment will be explained with reference to Figure 3. Figure 3 is a flowchart showing the flow of the processing control method S100 according to the first embodiment.

[0028] In step S101, the load prediction means 101 predicts the processing load of the data to be analyzed in the first processing unit 20.

[0029] In step S102, the bandwidth prediction means 102 predicts the communication bandwidth between the first processing unit 20 and the second processing unit 30.

[0030] In step S103, the switching control means 110 controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the processing load predicted by the load prediction means 101 and the communication bandwidth predicted by the bandwidth prediction means 102.

[0031] As described above, in the processing control method S100 according to this embodiment, the method controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the predicted processing load and communication bandwidth. Therefore, according to the processing control method S100 according to this embodiment, it is possible to switch between the analysis performed by the first processing unit 20 and the second processing unit 30 based on the communication bandwidth.

[0032] (Configuration of the processing control unit) The configuration of the processing control device 200 according to this embodiment will be described with reference to Figure 4. Figure 4 is a block diagram showing the configuration of the processing control device 200 according to the first embodiment. The processing control device 200 has a load prediction unit 201, a bandwidth prediction unit 202, and a switching control unit 210, and controls the processing system 1 (a first processing unit 20 that acquires data to be analyzed, and a second processing unit 30 that can communicate with the first processing unit 20).

[0033] The load prediction unit 201 has the same functionality as the load prediction means 101 and predicts the processing load of the data to be analyzed in the first processing unit 20. The bandwidth prediction unit 202 has the same functionality as the bandwidth prediction means 102 and predicts the communication bandwidth between the first processing unit 20 and the second processing unit 30. The switching control unit 210 has the same functionality as the switching control means 110 and controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the processing load predicted by the load prediction unit 201 and the communication bandwidth predicted by the bandwidth prediction unit 202.

[0034] The load prediction unit 201, the bandwidth prediction unit 202, and the switching control unit 210 may be computer devices in which processing is performed by the processor executing a program stored in memory. For example, the load prediction unit 201, the bandwidth prediction unit 202, and the switching control unit 210 may be a single computer device, or they may be a group of computer devices operating in cooperation with each other, or a group of server devices operating in cooperation with each other. Furthermore, at least a part of the load prediction unit 201, the bandwidth prediction unit 202, and the switching control unit 210 may be provided in the second processing unit 30. The processing control device 200 can achieve the same effects as the processing control system 100.

[0035] [Second Embodiment] A second embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0036] Figure 5 is a block diagram showing an example configuration of the processing control system 100 according to the second embodiment. The processing control system 100 includes a load prediction means 101, a bandwidth prediction means 102, and a switching control means 110, and controls the processing systems 1(1) and 1(2).

[0037] The processing system 1 according to this embodiment includes two processing systems 1(1) (imaging device 10(1), first processing unit 20(1), second processing unit 30(1)) and 1(2) (imaging device 10(2), first processing unit 20(2)), second processing unit 30(2)) which are controlled independently of each other by the processing control system 100. Here, two processing systems 1(1) and (2) are shown, but there may be three or more processing systems 1(i) (i: positive integer).

[0038] In this embodiment, the division of processing of the data to be analyzed between the first processing unit 20 and the second processing unit 30 is carried out as follows, for example. In the processing system 1(1), the first processing unit 20(1), controlled by the processing control system 100, processes at least a portion of the data to be analyzed D1 acquired from the imaging device 10(1) and transmits at least a portion of the data to be analyzed D1 to the second processing unit 30(1). At this time, the data to be analyzed transmitted to the second processing unit 30(1) is at least a portion of the data to be analyzed (the remainder of the data to be analyzed) for which all processing for analysis has not been completed in the first processing unit 20(1). For example, the first processing unit 20(1) transmits at least a portion of the remainder of the data to be analyzed D1 processed in the first processing unit 20(1) to the second processing unit 30(1). The second processing unit 30(1) receives and processes the data to be analyzed D1 transmitted from the first processing unit 20(1) (i.e., at least a portion of the data to be analyzed, for example, at least a portion of the remaining data to be analyzed that has not been processed by the first processing unit 20(1)). The first processing unit 20(1) also transmits at least a portion of the intermediate data (e.g., features) obtained as a result of processing the data to be analyzed in the first processing unit 20 to the second processing unit 30(1), and the second processing unit 30(1) may perform further processing on at least a portion of the received intermediate data. The processing in the processing system 1(2) is similar.

[0039] In other words, the data to be analyzed D1 output from the imaging device 10(1) is processed by the first processing unit 20(1) and the second processing unit 30(1). Similarly, the data to be analyzed D2 output from the imaging device 10(2) is also processed by the first processing unit 20(2) and the second processing unit 30(2).

[0040] Here, we assume that different bandwidths are allocated for transmitting the data to be analyzed D1 and D2 output from the imaging devices 10(1) and 10(2). For example, the amount of data to be analyzed per unit time, i.e., the bandwidth required for transmitting the data to be analyzed (e.g., the transfer rate), differs depending on the number of dots in the image of the data to be analyzed and the number of frames per unit time. Generally, the wider the bandwidth allocated for transmitting the data to be analyzed, the more important that data is considered to be. This is because a wide bandwidth is allocated for transmitting important data to be analyzed in order to increase the amount of information that can be obtained from important data to be analyzed.

[0041] Hereafter, for the sake of clarity, the imaging devices 10(1) and 10(2) may be referred to simply as imaging device 10 without distinction. Similarly, the first processing units 20(1) and 20(2) may be referred to as the first processing unit 20, and the second processing units 30(1) and 30(2) may be referred to as the second processing unit 30.

[0042] Figure 6 is a schematic diagram showing an example of data to be analyzed output from the imaging device 10. The data to be analyzed has multiple frames that are consecutive in time series. The first processing unit 20 and the second processing unit 30 process the data to be analyzed for each unit frame set, which consists of a predetermined number N frames and is the processing unit. Here, the frames in the unit frame set, which is the processing unit, are sequentially assigned numbers from 1 to N (a predetermined number). The predetermined number N is the number of frames that make up the unit frame set.

[0043] The first processing unit 20 and the second processing unit 30 process analysis target data for each unit frame set. As described above, the first processing unit 20 and the second processing unit 30 share the processing of the analysis target data. Therefore, while one of the first processing unit 20 and the second processing unit 30 is processing a unit frame set of the analysis target data, the processing of the analysis target data may be switched to the other. In this case, neither the first processing unit 20 nor the second processing unit 30 has data of the entire unit frame set, which makes it difficult to complete the processing of this unit frame set. For example, if processing is switched to the second processing unit 30 immediately after the first processing unit 20 processes the m-th (m<N) frame of the unit frame set, the processing results of the m frames processed by the first processing unit 20 may be wasted. Countermeasures against this will be described later.

[0044] The first processing unit 20 and the second processing unit 30 process analysis target data and extract feature quantities for each unit frame set. This feature quantity includes, for example, information for detecting and identifying analysis targets (objects, persons) in video. The first processing unit 20 and the second processing unit 30 perform tracking and time-series analysis of the analysis target based on the feature quantities, and for example, analyze the work content (e.g., ground preparation work, movement work) of a person (worker) and output the result as an analysis result. Note that the first processing unit 20 and the second processing unit 30 may extract feature quantities for each frame, perform analysis based on the feature quantities for each unit frame set, and output an analysis result.

[0045] Figure 7 shows an example of a video represented by the analysis target data. Here, in accordance with processing performed by the first processing unit 20 and the second processing unit 30, a video screen D is divided into a plurality of regions A. As described above, the first processing unit 20 or the second processing unit 30 may divide an image represented by the analysis target data into a plurality of regions based on the feature quantities, and analyze the work content of a person (worker) for each region.

[0046] The analysis results (work details), along with the analyzed video, can be displayed on a terminal held by a supervisor (for example, a site supervisor) via communication from the first processing unit 20 or the second processing unit 30. As a result, the supervisor can review the video of the work site along with the analysis results, accurately grasp the work situation, and give precise instructions to the site.

[0047] Here, the first processing unit 20 or the second processing unit 30 determines the reliability of the analysis result. This reliability is obtained by the processing result acquisition means 103, which will be described later. Reliability is an indicator of how confident the predicted analysis result is. When analyzing data using AI, the reliability of the analysis result can also be evaluated to make the analysis more reliable. In this case, the reliability parameter is output along with the analysis result. The first processing unit 20 or the second processing unit 30 may, for example, determine that if the reliability is high at a certain time, there is a high probability that the same analysis result will be output stably at the next time, and if the reliability is low, there is a high probability that a different analysis result will be output at the next time.

[0048] Figure 8 shows graphs G1 to G3, which represent examples of the prediction results of the communication bandwidth between the first processing unit 20 and the second processing unit 30 by the bandwidth prediction means 102. Graphs G1 to G3 each show examples of the temporal variation of the communication bandwidth from the present time. The upper and lower limits of the predicted communication bandwidth are shown as the upper limit bandwidth Bmax and the lower limit bandwidth Bmin. Over time, the upper limit bandwidth Bmax increases and the lower limit bandwidth Bmin decreases, and the range of the predicted bandwidth widens. This means that the certainty of the predicted bandwidth decreases as we move from the present time into the future.

[0049] Here, the temporal range of the prediction by the bandwidth prediction means 102 is sufficient to be from the present moment to the time (unit time) T corresponding to the unit frame set. This is because, since the processing of the data to be analyzed is done on a unit frame set basis, switching the processing of the data to be analyzed after unit time T does not affect the processing of the unit frame set currently being processed. In other words, the value of the lower limit bandwidth Bmin for a future unit time T can be used to decide whether to switch processing at the present moment.

[0050] Graphs G1 to G3 show that the predicted communication bandwidth decreases in order from G1 to G3. That is, the lower limit bandwidth Bmin after a unit time T decreases in the order of graphs G1 to G3. The predicted data amounts F1 and F2 represent the amount of data (transfer rate, i.e., bandwidth) that the second processing unit 30 is scheduled to process, and are, for example, the amount of data remaining after processing by the first processing unit 20 from the amount of data to be analyzed. Here, for the sake of clarity, the two predicted data amounts F1 and F2 are kept constant.

[0051] The switching control means 110 controls whether the data to be analyzed is performed by the first processing unit 20 or the second processing unit 30, based on the processing load predicted by the load prediction unit 201 and the communication bandwidth predicted by the bandwidth prediction means 102. For example, if the processing load predicted by the first processing unit 20 approaches the processing speed limit of the first processing unit 20 during analysis in the first processing unit 20, the switching control means 110 switches the analysis of the data to be analyzed from the first processing unit 20 to the second processing unit 30. For example, if the required processing speed of the data to be analyzed (e.g., predicted data amount F) approaches the predicted bandwidth (lower limit bandwidth Bmin) during analysis in the second processing unit 30, the switching control means 110 switches the analysis of the data to be analyzed from the second processing unit 30 to the first processing unit 20.

[0052] For example, in graph G1, the lower limit bandwidth Bmin expected after time T is greater than the predicted data amount F(F1, F2). Therefore, regardless of whether the predicted data amounts F1 or F2 are, the entire amount does not need to be analyzed by the first processing unit 20. Instead, it can be transmitted from the first processing unit 20 to the second processing unit 30 for analysis by the second processing unit 30. In graph G2, if the predicted data amount is F1, the entire amount can be analyzed by the first processing unit 20. However, if the predicted data amount is F2, it is difficult for the first processing unit 20 to analyze the entire amount. In graph G3, regardless of whether the predicted data amount is F1 or F2, it is difficult for the first processing unit 20 to analyze the entire amount. In such cases, it is conceivable to transmit the remaining portion of the data to be analyzed from the first processing unit 20 to the second processing unit 30, within the range that matches the lower limit bandwidth Bmin, for analysis.

[0053] Here, the switching control means 110 may determine which portion of the data to be analyzed will be discarded. In this case, the first processing unit 20 will not analyze this portion of the data and will not transmit it to the second processing unit 30. As a result, the portion of the data to be analyzed will be discarded. Note that discarding the data to be analyzed can be rephrased as not performing analysis on that data.

[0054] The switching control means 110 may determine which portions of the data to be analyzed to discard based on the communication bandwidth predicted by the bandwidth prediction means 102. For example, the switching control means 110 may decide to discard portions of the data to be analyzed (e.g., frames) whose data volume is greater than the sum of the predicted processing load and the predicted communication bandwidth.

[0055] The switching control means 110 may determine which parts of the data to be analyzed to discard based on the communication bandwidth allocated for transmitting the data to be analyzed D1 and D2. As described above, different bandwidths are allocated for transmitting the data to be analyzed D1 and D2 output from the imaging devices 10(1) and 10(2). For example, the switching control means 110 may determine that the parts of the data to be analyzed that have a large allocated communication bandwidth are of low importance and decide to discard them preferentially when the overall available bandwidth decreases.

[0056] With the above configuration, the analysis in the first processing unit 20 and the second processing unit 30 can be switched based on the communication bandwidth. Furthermore, the portion of the data to be analyzed to be discarded can be determined based on the predicted or allocated communication bandwidth.

[0057] Although the second embodiment has been described as a processing control system 100, the processing control system 100 according to the second embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the second embodiment may be the processing control method according to the second embodiment.

[0058] [Third Embodiment] A third embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0059] Figure 9 is a block diagram showing an example configuration of the processing control system 100 according to the third embodiment. The processing control system 100 includes a load prediction means 101, a bandwidth prediction means 102, a processing result acquisition means 103, and a switching control means 110, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the second embodiment in that the switching control means 110 determines the data to be analyzed to be discarded based on reliability.

[0060] The processing result acquisition means 103 obtains, for example, the reliability of the processing of the data to be analyzed from the first processing unit 20 or the second processing unit 30. As described above, the first processing unit 20 or the second processing unit 30 can analyze the data to be analyzed using AI and determine the reliability of the analysis results. The processing result acquisition means 103 can obtain the reliability of the processing of the data to be analyzed along with the analysis results from the first processing unit 20 or the second processing unit 30.

[0061] The switching control means 110 determines which parts of the data to be analyzed to discard based on the confidence level obtained by the processing result acquisition means 103. For example, the processing result acquisition means 103 determines that parts of the data to be analyzed that had a relatively high confidence level at a previous time are likely to yield the same results at the current time, and decides to interrupt processing and discard those parts of the data to be analyzed. As a result, the parts of the data to be analyzed with high confidence levels can be analyzed based on the analysis results from the previous time, while the parts of the data to be analyzed with low confidence levels can be analyzed based on the analysis results at the current time.

[0062] Although the third embodiment has been described as a processing control system 100, the processing control system 100 according to the third embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the third embodiment may be the processing control method according to the third embodiment.

[0063] [Fourth Embodiment] A fourth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0064] Figure 10 is a block diagram showing an example configuration of the processing control system 100 according to the fourth embodiment. The processing control system 100 includes a load prediction means 101, a bandwidth prediction means 102, an importance determination means 104, and a switching control means 110, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the second embodiment in that the switching control means 110 determines which data to discard based on importance.

[0065] The importance determination means 104 determines the importance of each part of the data to be analyzed. The importance is, for example, the priority of the processing of the data to be analyzed that is included in the data to be analyzed, and corresponds to the importance or risk level of the process shown in the data to be analyzed. The importance can be determined based on the AI ​​analysis of the detection or identification of the data to be analyzed in the first processing unit 20 or the second processing unit 30. Alternatively, the importance may be determined using a learning model that has learned the importance of the detection results of the data to be analyzed.

[0066] For example, the importance determination means 104 may determine the importance of each part by inputting input data, which is a combination of the feature quantities calculated for each part of the data to be analyzed, into a trained model. The trained model used may receive input data which is a combination of the feature quantities for each part, generate relationship information that shows the relationships between the feature quantities of each part based on the input data, and output the importance of each region based on the relationship information and the input data. In one embodiment, the relationship information shows the extent to which other regions other than the region in question are related to the importance of each region. In other words, the relationship information shows the relationships between regions such that, for each region, the relationship is large with respect to the regions necessary for determining the importance of the region in question, and small with respect to regions that are not necessary for determining the importance of a particular region. Examples of such relationship information include attention weights used in attention mechanisms such as self-attention mechanisms. The trained model may include, for example, one or more layers that generate relationship information based on input data, and one or more layers that generate the importance of each region based on the relationship information and the input data. A trained model can be trained, for example, by reinforcement learning using training input images labeled with analysis results and an analysis engine that analyzes the input images using importance levels.

[0067] The switching control means 110 determines which parts of the data to be analyzed to discard based on the importance determined by the importance determination means 104. For example, the switching control means 110 decides to process the parts of the data to be analyzed that are relatively important and discard the parts of the data to be analyzed that are relatively less important. This makes it possible to obtain analysis results based on the parts of the data to be analyzed that are of high importance.

[0068] Although the fourth embodiment has been described as a processing control system 100, the processing control system 100 according to the fourth embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the fourth embodiment may be the processing control method according to the fourth embodiment.

[0069] [Fifth Embodiment] A fifth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0070] Figure 11 is a block diagram showing an example configuration of the processing control system 100 according to the fifth embodiment. The processing control system 100 includes load prediction means 101, bandwidth prediction means 102, switching control means 110, and buffer control means 111, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the second embodiment in that it includes buffer control means 111.

[0071] The buffer control means 111 determines a number of buffer frames that is less than or equal to a predetermined number of frames constituting a unit frame set. By appropriately setting the number of buffer frames, the resources of the first processing unit 20 and the second processing unit 30 can be effectively utilized.

[0072] The buffer control means 111 buffers a number of frames equal to the buffer number of frames into the processing unit among the first processing unit 20 and the second processing unit 30 that is not analyzing the data to be analyzed. When a processing unit that is not processing the data to be analyzed switches to analyzing the data to be analyzed, the buffer control means 111 uses the buffered number of frames to analyze the data to be analyzed. This makes it possible to complete the analysis of a unit frame set using the buffered frames, even if the analysis switches in the middle of that unit frame set.

[0073] For example, when the second processing unit 30 is analyzing a frame, the first processing unit 20 buffers a number of frames equal to the buffer frame count. Then, when the analysis of the data to be analyzed switches from the second processing unit 30 to the first processing unit 20, the first processing unit 20 uses the buffered frames to analyze the data to be analyzed.

[0074] The buffer control means 111 may determine the number of buffer frames based on the communication bandwidth predicted by the bandwidth prediction means 102. For example, if the communication bandwidth is narrow, the number of buffer frames is increased, and if the communication bandwidth is wide, the number of buffer frames is decreased. This reduces frame loss even when the expected communication bandwidth is narrow.

[0075] The buffer control means 111 may determine the number of buffer frames based on the communication bandwidth allocated for transmitting the data to be analyzed. For example, if the allocated communication bandwidth is large, the number of buffer frames is increased, and if the allocated communication bandwidth is small, the number of buffer frames is decreased. This prevents the loss of processing of important portions of the data to be analyzed that are allocated a large amount of communication bandwidth.

[0076] Although the fifth embodiment has been described above as a processing control system 100, the processing control system 100 according to the fifth embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the fifth embodiment may be the processing control method according to the fifth embodiment.

[0077] [Sixth Embodiment] A sixth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0078] Figure 12 is a block diagram showing an example configuration of the processing control system 100 according to the sixth embodiment. The processing control system 100 includes load prediction means 101, bandwidth prediction means 102, processing result acquisition means 103, switching control means 110, and buffer control means 111, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the fifth embodiment in that the buffer control means 111 determines the number of buffer frames based on reliability.

[0079] As described above, the processing result acquisition means 103 acquires the reliability of the processing of the data to be analyzed, which has been determined by the first processing unit 20 or the second processing unit 30.

[0080] The buffer control means 111 determines the number of buffer frames based on the confidence level obtained by the processing result acquisition means 103. For example, the buffer control means 111 increases the number of buffer frames when the confidence level of processing the data to be analyzed is high, and decreases the number of buffer frames when the confidence level is low. This prevents the loss of highly reliable data to be analyzed.

[0081] Although the sixth embodiment has been described as a processing control system 100, the processing control system 100 according to the sixth embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the sixth embodiment may be the processing control method according to the sixth embodiment.

[0082] [Seventh Embodiment] A seventh embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0083] Figure 13 is a block diagram showing an example configuration of the processing control system 100 according to the seventh embodiment. The processing control system 100 according to the seventh embodiment includes load prediction means 101, bandwidth prediction means 102, importance determination means 104, switching control means 110, and buffer control means 111, and controls processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the fifth embodiment in that the buffer control means 111 determines the number of buffer frames based on importance.

[0084] As described above, the importance determination means 104 determines the importance of each part of the data to be analyzed. The buffer control means 111 determines the number of buffer frames based on the importance determined by the importance determination means 104. For example, the buffer control means 111 increases the number of buffer frames when the importance of the data to be analyzed is high, and decreases the number of buffer frames when the importance of the data to be analyzed is low. This prevents the loss of high-importance data to be analyzed.

[0085] Although the seventh embodiment has been described above as a processing control system 100, the processing control system 100 according to the seventh embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the seventh embodiment may be the processing control method according to the seventh embodiment.

[0086] [Eighth Embodiment] An eighth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0087] Figure 14 is a block diagram showing an example configuration of the processing control system 100 according to the eighth embodiment. The processing control system 100 according to the eighth embodiment includes load prediction means 101, bandwidth prediction means 102, switching control means 110, complementary control means 112, and data storage means 115, and controls processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the second embodiment in that it includes complementary control means 112.

[0088] The interpolation control means 112 instructs the first processing unit 20 and the second processing unit 30 to interpolate the frames that were processed before the switch when the system switches from processing data to be analyzed to processing data to be analyzed. This allows the system to analyze the unit frame set even if the analysis switches in the middle of the unit frame set by interpolating the frames.

[0089] The data storage means 115 may be located outside the second processing unit 30 and may hold the processing results of the second processing unit 30. Alternatively, the processing results of the second processing unit 30 may be held by the second processing unit 30 itself instead of the data storage means 115. For the sake of clarity, in the following, we will not consider the presence or absence of the data storage means 115 as relevant, and will simply state that the second processing unit 30 holds the processing results.

[0090] Similarly, the data storage means 115 may be located outside the first processing unit 20 and hold the processing results of the first processing unit 20. Alternatively, the processing results of the first processing unit 20 may be held by the first processing unit 20 itself instead of the data storage means 115. For the sake of clarity, in the following, we will not consider the presence or absence of the data storage means 115 as relevant, and will simply state that the first processing unit 20 holds the processing results.

[0091] Here, the following methods can be listed as techniques for completing a unit frame set: (1) duplication and (2) extraction. In the case of duplication, the data storage means 115 is not required.

[0092] (1) The interpolation control means 112 interpolates the frames that were processed before the switch by duplicating the frame to be processed first after the switch. For example, consider the case where the processing of the data to be analyzed is switched to the second processing unit 30 immediately after the first processing unit 20 has processed the i-th frame of the unit frame set. In this case, the second processing unit 30 interpolates the unit frame set by duplicating the i+1-th frame to be processed first after the switch, making them frames 1 to i, and then processes the unit frame set. This ensures that the unit frame set can be analyzed reliably.

[0093] Similarly, consider the case where, immediately after the second processing unit 30 has processed the i-th frame of the unit frame set, the analysis of the data to be analyzed is switched to the first processing unit 20. In this case, the first processing unit 20 duplicates the i+1-th frame, which is the first frame to be processed after the switch, to make them frames 1 through i, thereby complementing the unit frame set and analyzing the unit frame set. This ensures that the unit frame set is analyzed reliably.

[0094] (2) The first processing unit 20 and the second processing unit 30 retain the processing results of the frames. The interpolation control means 112 extracts processing results from these retained processing results that are similar to the processing results of the frame to be processed first after the switch, thereby interpolating the frames that were processed before the switch. For example, consider the case where the processing of the data to be analyzed is switched to the second processing unit 30 immediately after the first processing unit 20 has processed the i-th frame of the unit frame set. In this case, the second processing unit 30 extracts past processing results similar to the i+1-th frame to be processed first after the switch, and uses these as frames 1 to i, thereby interpolating and analyzing the frames that make up the unit frame set. This ensures that the unit frame set can be reliably analyzed.

[0095] Similarly, consider the case where, immediately after the second processing unit 30 has processed the i-th frame of the unit frame set, the analysis of the data to be analyzed is switched to the first processing unit 20. In this case, the first processing unit 20 extracts past processing results similar to the i+1-th frame that is processed first after the switch, and uses these as frames 1 through i, thereby complementing and analyzing the frames that make up the unit frame set. This ensures that the unit frame set can be analyzed reliably.

[0096] Although the eighth embodiment has been described above as a processing control system 100, the processing control system 100 according to the eighth embodiment may be mounted on a single device as a processing control device. Furthermore, the operation of the processing control system 100 according to the eighth embodiment may be the processing control method according to the eighth embodiment.

[0097] [Ninth Embodiment] A processing control system 100 according to the ninth embodiment of the present invention will be described in detail. The processing control system 100 according to the ninth embodiment will be described with reference to Figure 14.

[0098] In the ninth embodiment of the present invention, the interpolation control means 112 determines an upper limit number of frames, which is the upper limit of the number of frames to be interpolated. If the number of frames to be interpolated exceeds the upper limit number of frames, the interpolation control means 112 does not interpolate the frames that were processed before the switchover.

[0099] For example, consider a case where the upper limit of the number of frames is p, and immediately after the first processing unit 20 processes the i-th frame of the unit frame set, the analysis of the data to be analyzed is switched to the second processing unit 30. In this case, the number of frames to be filled in is "Ni", which is the value obtained by subtracting i from a predetermined number N of unit frame sets. If the number of frames to be filled in "Ni" is less than or equal to p, the second processing unit 30 fills in the frames and analyzes the unit frame set. If the number of frames to be filled in "Ni" exceeds p, the second processing unit 30 does not fill in the frames and does not analyze this unit frame set.

[0100] Similarly, consider the case where the upper limit of the number of frames is p, and immediately after the second processing unit 30 and the first processing unit 20 process the i-th frame of the unit frame set, the analysis of the data to be analyzed is switched to the first processing unit 20. In this case, the number of frames to be interpolated is "Ni", which is the value obtained by subtracting i from a predetermined number N of unit frame sets. If the number of frames to be interpolated "Ni" is less than or equal to p, the first processing unit 20 interpolates the frames and analyzes the unit frame set. If the number of frames to be interpolated "Ni" exceeds p, the first processing unit 20 does not interpolate the frames and does not analyze this unit frame set.

[0101] Here, the interpolation control means 112 may determine the upper limit of the number of frames based on the allocated communication bandwidth. For example, if the communication bandwidth allocated for transmitting the data portion to be analyzed is large, the interpolation control means 112 will increase the upper limit of the number of frames, and if the communication bandwidth allocated for transmitting the data portion to be analyzed is small, it will decrease the upper limit of the number of frames. This prevents the loss of analysis of important data portions to be analyzed that have a large allocated communication bandwidth.

[0102] Although the ninth embodiment has been described above as a processing control system 100, the processing control system 100 according to the ninth embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the ninth embodiment may be the processing control method according to the ninth embodiment.

[0103] [Tenth Embodiment] A tenth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0104] Figure 15 is a block diagram showing an example configuration of the processing control system 100 according to the tenth embodiment. The processing control system 100 includes a load prediction means 101, a bandwidth prediction means 102, a processing result acquisition means 103, a switching control means 110, and a supplementation control means 112, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the eighth embodiment in that the supplementation control means 112 determines the upper limit number of frames for supplementation based on reliability.

[0105] The processing result acquisition means 103 acquires the reliability of the processing of the data to be analyzed, and the interpolation control means 112 determines the upper limit number of frames based on the reliability acquired by the processing result acquisition means 103. For example, if the reliability of the processing of the data to be analyzed is high, the interpolation control means 112 increases the upper limit number of frames, and if the reliability is low, it decreases the upper limit number of frames. This prevents the loss of highly reliable data to be analyzed.

[0106] Although the tenth embodiment has been described above as a processing control system 100, the processing control system 100 according to the tenth embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the tenth embodiment may be the processing control method according to the tenth embodiment.

[0107] [Embodiment 11] An eleventh embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0108] Figure 16 is a block diagram showing an example configuration of the processing control system 100 according to the eleventh embodiment. The processing control system 100 includes a load prediction means 101, a bandwidth prediction means 102, an importance determination means 104, a switching control means 110, a supplementation control means 112, and a data storage means 115, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the eighth embodiment in that the supplementation control means 112 determines the upper limit number of frames to be supplemented based on importance.

[0109] The importance determination means 104 determines the importance of each part of the data to be analyzed, and the interpolation control means 112 determines the upper limit of the number of frames based on the importance determined by the importance determination means 104. For example, if the importance of the data to be analyzed is high, the interpolation control means 112 increases the upper limit of the number of frames, and if the importance of the data to be analyzed is low, it decreases the upper limit of the number of frames. This prevents the loss of highly important data to be analyzed.

[0110] Although the 11th embodiment has been described as a processing control system 100, the processing control system 100 according to the 11th embodiment may be implemented as a processing control device mounted on a single device. Furthermore, the operation of the processing control system 100 according to the 9th embodiment may be the processing control method according to the 11th embodiment.

[0111] [Twelfth Embodiment] A twelfth embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in the first embodiment will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0112] Figure 17 is a block diagram showing an example configuration of the processing control system 100 according to the twelfth embodiment. The processing control system 100 includes load prediction means 101, bandwidth prediction means 102, learning means 105, switching control means 110, buffer / completion control means 113, and data storage means 115, and controls the processing systems 1(1) and 1(2). The processing control system 100 according to this embodiment differs from the fifth and eighth embodiments in that it includes the buffer / completion control means 113.

[0113] The buffer / completion control means 113 has the combined function of the buffer control means 111 and the complementation control means 112, and can switch between buffering control of the data to be analyzed by the buffer control means 111 and complementation control of the data to be analyzed by the complementation control means 112. The processing control system 100 may have the buffer control means 111 and the complementation control means 112 instead of the buffer / completion control means 113.

[0114] The learning means 105 learns, based on the communication bandwidth predicted by the bandwidth prediction means 102, whether to use buffering control or interpolation control, and how to determine the number of buffer frames in buffering control and the upper limit number of frames in interpolation control. Based on these learning results, the learning means 105 selects between buffering control and interpolation control.

[0115] Although the twelfth embodiment has been described above as a processing control system 100, the processing control system 100 according to the twelfth embodiment may be mounted on a single device as a processing control device. Furthermore, the operation of the processing control system 100 according to the twelfth embodiment may be the processing control method according to the twelfth embodiment.

[0116] This disclosure is not limited to the embodiments described above, and various modifications are possible. Embodiments obtained by appropriately combining the configurations, operations, and processes disclosed in different embodiments are also included in the technical scope of this disclosure. Furthermore, embodiments obtained by appropriately changing the order of operations and processes disclosed in each embodiment are also included in the technical scope of this disclosure.

[0117] Each configuration according to the first to twelfth embodiments may be implemented by (1) one or more hardware components, (2) one or more software components, (3) a combination of hardware and software components, or (4) a cloud server. Each device, each function, and each process may be implemented by at least one computer having at least one processor and at least one memory. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 18. For example, a program for implementing the processing control method described in the first to twelfth embodiments may be stored in memory C2, and each function described in the first to twelfth embodiments may be implemented by processor C reading and executing the program P stored in memory C2.

[0118] Program P, when loaded into computer C, includes a set of instructions that cause computer C to execute one or more functions described in the first to twelfth embodiments. Program P is stored in memory C2. For example, a CPU (Central Processing Unit) can be used as the processor C1. For example, a Read Only Memory (ROM), Random Access Memory (RAM), flash memory, or Solid State Drive (SSD) can be used as the memory 1602.

[0119] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0120] This disclosure is not limited to the embodiments described above. That is, the present invention can be applied in various forms that can be understood by those skilled in the art within the scope of this disclosure. Some or all of the embodiments described above may also be described below. However, the present invention is not limited to the forms described below.

[0121] (Note 1) A processing control system that controls a first processing unit that acquires data to be analyzed from an imaging device, and a second processing unit that can communicate with the first processing unit, The first processing unit processes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit processes at least a portion of the data to be analyzed transmitted from the first processing unit, The processing control system is Load prediction means for predicting the processing load of the data to be analyzed in the first processing unit, Bandwidth prediction means for predicting the communication bandwidth between the first processing unit and the second processing unit, A switching control means controls which of the first processing unit and the second processing unit processes the data to be analyzed, based on the predicted processing load and the predicted communication bandwidth. A processing control system equipped with the following features.

[0122] (Note 2) The switching control means determines, based on the predicted communication bandwidth, the portion of the data to be analyzed to be discarded, as described in Appendix 1, in the processing control system.

[0123] (Note 3) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The processing control system comprises buffer control means, The buffer control means is Determine the number of buffer frames that is less than or equal to the predetermined number, The processing control system according to Appendix 1, wherein the processing unit among the first and second processing units that is not processing the data to be analyzed buffers the number of buffer frames, and when the processing unit switches to processing the data to be analyzed, it analyzes the data using the buffered number of buffer frames.

[0124] (Note 4) The buffer control means determines the number of buffer frames based on the predicted communication bandwidth, as described in Appendix 3 of the processing control system.

[0125] (Note 5) The processing control system includes importance determination means for determining the importance of each part of the data to be analyzed, The buffer control means determines the number of buffer frames based on the importance, as described in Appendix 3 of the processing control system.

[0126] (Note 6) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The processing control system comprises complementary control means, The processing control system according to Appendix 1, wherein the complementary control means causes the first processing unit and the second processing unit to complement the frames that were being processed in a unit frame set before the switch when they switch from a state in which they are not processing the data to be analyzed to a state in which they are processing the data to be analyzed.

[0127] (Note 7) The processing control system described in Appendix 6, wherein the complementary control means complements the frames that were processed before the switch by duplicating the frame to be processed first after the switch.

[0128] (Note 8) A processing control device that controls a first processing unit that acquires data to be analyzed from an imaging device, and a second processing unit that can communicate with the first processing unit, The first processing unit processes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit processes at least a portion of the data to be analyzed transmitted from the first processing unit, The aforementioned processing control device is The load prediction unit predicts the processing load of the data to be analyzed in the first processing unit, A bandwidth prediction unit predicts the communication bandwidth between the first processing unit and the second processing unit, A switching control unit controls which of the first processing unit and the second processing unit processes the data to be analyzed, based on the predicted processing load and the predicted communication bandwidth. A processing control device equipped with the following:

[0129] (Note 9) The switching control unit determines, based on the predicted communication bandwidth, the portion of the data to be analyzed to be discarded, as described in Appendix 8, for processing control devices.

[0130] (Note 10) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The aforementioned processing control device comprises a buffering control unit, The buffering control unit, Determine the number of buffer frames that is less than or equal to the predetermined number, The processing control device according to Appendix 8, wherein, among the first and second processing units, the processing unit that is not processing the data to be analyzed buffers the number of buffer frames, and when the processing unit switches to processing the data to be analyzed, it analyzes the data using the buffered number of buffer frames.

[0131] (Note 11) The buffering control unit determines the number of buffer frames based on the predicted communication bandwidth, as described in Appendix 10.

[0132] (Note 12) The processing control device includes an importance determination unit that determines the importance of each part of the data to be analyzed, The buffering control unit determines the number of buffer frames based on the importance, as described in Appendix 10.

[0133] (Note 13) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The aforementioned processing control device is The processing control device according to Appendix 8, wherein the first processing unit and the second processing unit are equipped with a complementary processing control unit that, when switching from a state in which the data to be analyzed is not being processed to a state in which the data to be analyzed is being processed, complements the frames that were processed in a unit frame set before the switch.

[0134] (Note 14) The processing control device described in Appendix 13, wherein the interpolation processing control unit interpolates the frame that was processed before the switching by duplicating the frame to be processed first after the switching.

[0135] (Note 15) Processing control that controls a first processing unit and a second processing unit that can communicate with the first processing unit and shares the task of analyzing the data to be analyzed with the first processing unit. method And, The load prediction process in the first processing unit predicts the processing load of the data to be analyzed, A bandwidth prediction process that predicts the communication bandwidth between the first processing unit and the second processing unit, Based on the predicted processing load and the predicted communication bandwidth, a switching control process controls which of the first processing unit and the second processing unit will analyze the data to be analyzed. Execute, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit is a processing control method that analyzes at least a portion of the data to be analyzed transmitted from the first processing unit.

[0136] (Note 16) The processing control method described in Appendix 15, wherein the switching control means determines, based on the predicted communication bandwidth, the portion of the data to be analyzed to be discarded.

[0137] (Note 17) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The processing control method is, The buffer control means determines a number of buffer frames that is less than or equal to the predetermined number, and The buffer control means causes the first and second processing units, which are not processing the data to be analyzed, to buffer the buffer frame number of frames, and when the processing unit switches to processing the data to be analyzed, it uses the buffered buffer frame number of frames to analyze the data to be analyzed. The processing control method described in Appendix 15, including the method described therein.

[0138] (Note 18) The processing control method described in Appendix 17, wherein the buffer control means determines the number of buffer frames based on the predicted communication bandwidth.

[0139] (Note 19) The processing control method is, The importance determination means determines the importance of each part of the data to be analyzed, and The buffer control means determines the number of buffer frames based on the importance. The processing control method described in Appendix 17, including the method described in Appendix 17.

[0140] (Note 20) The aforementioned data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The processing control method described in Appendix 15, wherein the interpolation control means causes the first processing unit and the second processing unit to interpolate frames that were processed in a unit frame set before the switching when the state of not processing the data to be analyzed is switched to processing the data to be analyzed.

[0141] (Note 21) A communication bandwidth is allocated for transmitting the aforementioned data to be analyzed. The switching control means determines, based on the allocated communication bandwidth, the portion of the data to be analyzed to be discarded, as described in Appendix 1, in the processing control system.

[0142] (Note 22) The processing control system includes processing result acquisition means for acquiring the reliability of processing the data to be analyzed, The switching control means determines, based on the reliability, the portion of the data to be analyzed to be discarded, as described in Appendix 1 of the processing control system.

[0143] (Note 23) The processing control system includes importance determination means for determining the importance of each part of the data to be analyzed, The switching control means determines, based on the importance, the portion of the data to be analyzed to be discarded from the data to be analyzed, as described in Appendix 1.

[0144] (Note 24) A communication bandwidth is allocated for transmitting the aforementioned data to be analyzed. The buffer control means determines the number of buffer frames based on the allocated communication bandwidth, as described in Appendix 3 of the processing control system.

[0145] (Note 25) The processing control system includes processing result acquisition means for acquiring the reliability of processing the data to be analyzed, The buffer control means determines the number of buffer frames based on the reliability, as described in Appendix 3 of the processing control system.

[0146] (Note 26) The first processing unit and the second processing unit hold the processing results of the frame, The processing control system according to Appendix 6, wherein the complementary control means complements the frames that were processed before the switch by extracting from the retained processing results a processing result similar to the processing result of the frame to be processed first after the switch.

[0147] (Note 27) The aforementioned complementary control means is Determine the upper limit of the number of frames to be interpolated. The processing control system described in Appendix 6, which, if the number of frames to be supplemented exceeds the upper limit number of frames, does not supplement the frames that were processed before the switchover.

[0148] (Note 28) A communication bandwidth is allocated for transmitting the aforementioned data to be analyzed. The processing control system according to Appendix 27, wherein the complementary control means determines the upper limit number of frames based on the allocated communication bandwidth.

[0149] (Note 29) The processing control system includes processing result acquisition means for acquiring the reliability of processing the data to be analyzed, The processing control system described in Appendix 27, wherein the complementary control means determines the upper limit number of frames based on the reliability.

[0150] (Note 30) The processing control system includes importance determination means for determining the importance of each part of the data to be analyzed, The processing control system described in Appendix 27, wherein the complementary control means determines the upper limit number of frames based on the importance.

[0151] (Note 31) The processing control system described above can also be expressed as follows:

[0152] A processing control system that controls a first processing unit and a second processing unit that can communicate with the first processing unit, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, The processing control system is It comprises at least one processor, the processor being, The load prediction process in the first processing unit predicts the processing load of the data to be analyzed, A bandwidth prediction process that predicts the communication bandwidth between the first processing unit and the second processing unit, Based on the predicted processing load and the predicted communication bandwidth, a switching control process controls which of the first processing unit and the second processing unit will analyze the data to be analyzed. A processing control system that executes [this].

[0153] Furthermore, this processing control system may also include at least one memory, which may store a program that causes the processor to execute the load prediction process, the bandwidth prediction process, and the switching control process. This program may also be recorded on a computer-readable, non-temporary, tangible recording medium.

[0154] (Note 32) The processing control device described above can also be expressed as follows:

[0155] A processing control device that controls a first processing unit and a second processing unit that can communicate with the first processing unit, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, The aforementioned processing control device is It comprises at least one processor, the processor being, The load prediction process in the first processing unit predicts the processing load of the data to be analyzed, A bandwidth prediction process that predicts the communication bandwidth between the first processing unit and the second processing unit, Based on the predicted processing load and the predicted communication bandwidth, a switching control process controls which of the first processing unit and the second processing unit will analyze the data to be analyzed. A processing control device that executes this process.

[0156] Furthermore, this processing control device may also include at least one memory, which may store a program that causes the processor to execute the load prediction process, the bandwidth prediction process, and the switching control process. This program may also be recorded on a computer-readable, non-temporary, tangible recording medium. [Explanation of Symbols]

[0157] 100 Processing Control System 101 Load prediction means 102 Bandwidth prediction means 103 Processing result acquisition means 104 Importance determination means 110 Switching control means 111 Buffer control means 112 Complementary control means 113 Buffer / Complementation Control Means 115 Data storage means

Claims

1. A processing control system that controls a first processing unit and a second processing unit that can communicate with the first processing unit, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, The data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The processing control system is Load prediction means for predicting the processing load of the data to be analyzed in the first processing unit, A bandwidth prediction means for predicting the communication bandwidth between the first processing unit and the second processing unit, A switching control means controls which of the first processing unit and the second processing unit will analyze the data to be analyzed, based on the predicted processing load and the predicted communication bandwidth. The first processing unit and the second processing unit are provided with a interpolation control means that, when switching from a state where the data to be analyzed is not being processed to a state where the data to be analyzed is being processed, interpolates the frames that were processed in the unit frame set before the switch. A processing control system equipped with the following features.

2. The processing control system according to claim 1, wherein the switching control means determines the portion of the data to be analyzed to be discarded based on the predicted communication bandwidth.

3. The processing control system includes processing result acquisition means for acquiring the reliability of processing the data to be analyzed, The processing control system according to claim 1 or 2, wherein the switching control means determines, based on the reliability, the portion of the data to be analyzed to be discarded from the data to be analyzed.

4. The processing control system includes importance determination means for determining the importance of each part of the data to be analyzed, The processing control system according to claim 1 or 2, wherein the switching control means determines the portion of the data to be analyzed to be discarded based on the importance level.

5. The processing control system according to claim 1, wherein the complementary control means complements the frames that were processed before the switch by duplicating the frame to be processed first after the switch.

6. A communication bandwidth is allocated for transmitting the aforementioned data to be analyzed. The aforementioned complementary control means is Based on the allocated communication bandwidth, the upper limit of the number of frames to be supplemented is determined. The processing control system according to claim 1 or 5, wherein if the number of frames to be supplemented exceeds the upper limit number of frames, the frame that was processed before the switchover is not supplemented.

7. The processing control system includes processing result acquisition means for acquiring the reliability of processing the data to be analyzed, The aforementioned complementary control means is Based on the aforementioned reliability, the upper limit of the number of frames to be interpolated is determined, The processing control system according to claim 1 or 5, wherein if the number of frames to be supplemented exceeds the upper limit number of frames, the frame that was processed before the switchover is not supplemented.

8. A processing control device that controls a first processing unit and a second processing unit that can communicate with the first processing unit, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, The data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. The aforementioned processing control device is The first processing unit includes a load prediction unit that predicts the processing load of the data to be analyzed, A bandwidth prediction unit predicts the communication bandwidth between the first processing unit and the second processing unit, A switching control unit controls which of the first processing unit and the second processing unit will analyze the data to be analyzed, based on the predicted processing load and the predicted communication bandwidth. The first processing unit and the second processing unit include a interpolation control unit that, when switching from a state where the data to be analyzed is not being processed to a state where the data to be analyzed is being processed, interpolates the frames that were processed in the unit frame set before the switch, A processing control device equipped with the following:

9. A processing control method for controlling a first processing unit and a second processing unit that can communicate with the first processing unit, The load prediction process in the first processing unit predicts the processing load of the data to be analyzed, A bandwidth prediction process that predicts the communication bandwidth between the first processing unit and the second processing unit, Based on the predicted processing load and the predicted communication bandwidth, a switching control process controls which of the first processing unit and the second processing unit will analyze the data to be analyzed. The first processing unit and the second processing unit are provided with a interpolation control process that, when switching from a state where the data to be analyzed is not being processed to a state where the data to be analyzed is being processed, interpolates the frames that were processed before the switch in a unit frame set. Execute, The first processing unit analyzes at least a portion of the data to be analyzed and transmits at least a portion of the data to be analyzed to the second processing unit. The second processing unit analyzes at least a portion of the data to be analyzed transmitted from the first processing unit, The data to be analyzed has multiple frames that are consecutive in time series, The first processing unit and the second processing unit process the data to be analyzed for each unit frame set consisting of a predetermined number of frames, which is a processing unit. Processing control method.

Images