Data processing device and imaging device

Abstract

To achieve the efficiency of data processing.SOLUTION: A data processing device includes a first module for executing first data processing for data, a second module for executing second data processing for the data after the start of the first data processing, and a sequencer for controlling one of the first module and the second module on the basis of a difference between a first position where the data is processed by the first data processing and a second position where the data is processed by the second data processing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data processing device and an imaging device. [Background Art]

[0002] Conventionally, it has been disclosed that image processing is performed by an image processing apparatus in which a plurality of image processing units are directly connected and connected. [Prior Art Literature] [Patent Literature]

[0003] [Patent Literature 1] Japanese Unexamined Patent Publication No. 2010-263475 [Summary of the Invention]

[0004] A data processing device according to a first disclosure technique comprises: 1 a first module that executes first data processing on data; and after the start of said first data processing, said 1 a second module that executes second data processing on data; and said first data processing causes said 1 a first position at which data is processed and said second data processing causes said 1 either one of said first module and said second module to be controlled based on a difference between said second position at which data is processed 1 a sequencer; The first sequencer controls the first module to temporarily suspend the first data processing when the difference becomes greater than or equal to a first threshold.

[0005] A data processing device according to a second disclosure technique comprises: a module that processes data; and a sequencer that changes a priority at which said module accesses said data if a position at which said data is processed is outside a predetermined range.

[0006] The data processing device, which is the third disclosed technology, includes a module that processes data and specifies a logical address in a logical address space to which the data is accessed; a memory having an area in which the data is stored; and a conversion unit that converts the logical address specified by the module to a physical address within the physical address range of the memory that cycles from the upper physical address to the lower physical address, and accesses the converted physical address. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a block diagram showing an example of the hardware configuration of a data processing device. [Figure 2] Figure 2 is a block diagram showing the connection relationship between the i-th sequencer and the module group. [Figure 3] Figure 3 is a block diagram showing a specific example of the connection relationship between the i-th sequencer and the module group. [Figure 4] Figure 4 is an explanatory diagram showing an example of overtaking prevention control. [Figure 5] Figure 5 is an explanatory diagram showing an example of buffer overflow prevention control. [Figure 6] Figure 6 is an explanatory diagram illustrating an example of how to start a group of modules using host control. [Figure 7] Figure 7 is an explanatory diagram illustrating an example of how to start a group of modules using KICKVD control. [Figure 8] Figure 8 is an explanatory diagram illustrating an example of the coordinated operation between one pre-stage module and two post-stage modules. [Figure 9] Figure 9 is an explanatory diagram illustrating an example of the coordinated operation between two pre-stage modules and one post-stage module. [Figure 10] Figure 10 is a graph showing an example of virtual line counter-following QoS priority control. [Figure 11] Figure 11 is an explanatory diagram showing an example of a region setting table. [Figure 12]Figure 12 is an explanatory diagram illustrating an example of address space remapping. [Modes for carrying out the invention]

[0008] <Example of data processing device hardware configuration> Figure 1 is a block diagram showing an example of the hardware configuration of a data processing device. The data processing device 100 is a device that performs data processing such as image data, and includes, for example, a sequencer set 101, a module set 102, an interconnect 103, a DRAM (Dynamic Random Access Memory) 104, an SRAM system 105, and a processor 106.

[0009] The sequencer set 101 controls data processing by the module set 102 in a predetermined order. The sequencer set 101 is a set of sequencers 101-1 to nth sequencers 101-n (where n is an integer greater than or equal to 2). If the sequencers 101-1 to nth sequencers 101-n are not distinguished, they are referred to as the ith sequencer 101-i (where i is an integer satisfying 1 ≤ i ≤ n). The ith sequencer 101-i controls the execution order of data processing by each module of the connected module group Mi.

[0010] Module set 102 is a collection of multiple module groups M1 to Mn. When module groups M1 to Mn are not distinguished, they are referred to as module group Mi. Module group Mi contains multiple modules (details are described later in Figure 2). Module group Mi directly accesses the DRAM 104 and SRAM system 105 and performs data processing under control from the i-th sequencer 101-i that it connects to.

[0011] The interconnect 103 is a data transmission path between the module assembly 102 and the DRAM 104, and between the module assembly 102 and the SRAM system 105.

[0012] DRAM 104 is a volatile memory connected to the interconnect 103 and that retains data. DRAM 104 is a memory that is accessed from the module group Mi at a lower speed than SRAM 152.

[0013] The SRAM system 105 is connected to the interconnect 103. The SRAM system 105 includes a conversion unit 151, an SRAM 152, and an area setting table 153. The conversion unit 151 is a circuit that remaps an address space and controls the module group Mi to access the address of the SRAM 152 after remapping.

[0014] SRAM 152 is a volatile memory that retains data. SRAM 152 is a memory that is accessed from the module group Mi at a higher speed than DRAM 104.

[0015] The area setting table 153 is a table for setting accessible areas of the SRAM 152 for each module group Mi.

[0016] The processor 106 outputs an operation start signal to the i-th sequencer 101-i, and updates the area setting table 153. The processor 106 also outputs an ACT signal that activates the module group Mi.

[0017] <Connection Relationship Between i-th Sequencer 101-i and Module Group Mi> FIG. 2 is a block diagram showing the connection relationship between the i-th sequencer 101-i and the module group Mi. The module group Mi is a set of one or more modules Mi1 to Mim, where m is an integer of 1 or more. When the modules Mi1 to Mim are not distinguished from each other, they are denoted as a module Mij, where j is an integer satisfying 1≦j≦m.

[0018] In module set 102, there may be a module group Mi with one module Mij, but module set 102 may contain one or more module groups Mi with two or more modules Mij. The i-th sequencer 101-i is connected to each module Mij and controls data processing by the modules Mij in a predetermined order.

[0019] Figure 3 is a block diagram showing a specific example of the connection relationship between the i-th sequencer 101-i and the module group Mi. Figure 3 shows a specific example of the connection relationship between the i-th sequencer 101-i and the module group Mi when j=2. The i-th sequencer 101-i is connected to the preceding module Mi1 (module A in Figure 3) and the following module Mi2 (module B in Figure 3).

[0020] The i-th sequencer 101-i controls the data to be processed, such as image data, to be processed using the preceding module A(Mi1) before the subsequent module B(Mi2), and then to be processed using the subsequent module B(Mi2). Specifically, for example, the preceding module A(Mi1) reads the image data sequentially from the first line and performs processing specific to the preceding module A(Mi1), and the subsequent module B(Mi2) reads the lines of image data that the preceding module A(Mi1) has processed and performs processing specific to the subsequent module B(Mi2).

[0021] The preceding module A (Mi1) and the succeeding module B (Mi2) each output a VD signal, an LE signal, and an FE signal to the i-th sequencer 101-i. The VD signal is a signal that each of the preceding module A (Mi1) and the succeeding module B (Mi2) uses to notify the start of data processing.

[0022] The LE signal is a line-end signal output when each of the preceding module A (Mi1) and the succeeding module B (Mi2) has completed processing one line of image data. The FE signal is a frame-end signal output when each of the preceding module A (Mi1) and the succeeding module B (Mi2) has completed processing one frame of image data.

[0023] The i-th sequencer 101-i outputs the FLLW_KICKVD signal and the FLLW_NOGO signal to the downstream module B (Mi2), and outputs the LEAD_NOGO signal to the preceding module A (Mi1). The FLLW_KICKVD signal is an operation start signal that causes the i-th sequencer 101-i to start data processing in the downstream module B (Mi2). The FLLW_NOGO signal is a pause signal that causes the i-th sequencer 101-i to temporarily suspend data processing in the downstream module B (Mi2). The LEAD_NOGO signal is a pause signal that causes the i-th sequencer 101-i to temporarily suspend data processing in the preceding module A (Mi1).

[0024] If the LEAD_NOGO signal and FLLW_NOGO signal are not distinguished, they are simply referred to as the NOGO signal. From the rising edge to the falling edge of the NOGO signal, module Mij receives the NOGO signal and stops processing. On the rising edge of the NOGO signal, module Mij resumes processing.

[0025] <Overtaking prevention control> Next, we will explain overtaking prevention control. Overtaking prevention control is a control method in which the i-th sequencer 101-i prevents the subsequent module B (Mi2) from overtaking the processing of the preceding module A (Mi1). Specifically, for example, overtaking prevention control means that the i-th sequencer 101-i controls so that the number of lines of image data processed by the subsequent module B (Mi2) does not exceed the number of lines of image data processed by the preceding module A (Mi1).

[0026] Figure 4 is an explanatory diagram showing an example of overtaking prevention control. The upper part of Figure 4 is a graph 400 showing the temporal change in the position where the preceding module A (Mi1) and the subsequent module B (Mi2) processed the data, and the lower part is a timing chart 410 showing the control by the i-th sequencer 101-i.

[0027] In Graph 400, the horizontal axis represents time, and the vertical axis represents the LE counter value counted by the i-th sequencer 101-i. The LE counter value is the count of the number of LE signal inputs from the preceding module A (Mi1) and the succeeding module B (Mi2), respectively, and indicates the position where the data has been processed. The FE on the vertical axis represents the count of LE signal inputs when the FE signal is input. That is, when FE is reached, module Mij, the source of the FE signal output, completes the processing of one frame of data.

[0028] In Graph 400, the solid line 401a shows the temporal change in the actual LE counter value in the preceding module A (Mi1), where the LE counter value increases monotonically over time. The solid line 402a shows the temporal change in the actual LE counter value in the subsequent module B (Mi2). The dotted line 420a shows the temporal change in the ideal LE counter value in the subsequent module B (Mi2), where the LE counter value increases monotonically over time.

[0029] Assume that at time t0, the i-th sequencer 101-i outputs a LEAD_KICKVD signal to the pre-stage module A(Mi1), and the pre-stage module A(Mi1) starts up. The pre-stage module A(Mi1) processes the image data as shown by solid line 401a.

[0030] At time t1, the i-th sequencer 101-i outputs the FLLW_KICKVD signal to the subsequent module B (Mi2), and the subsequent module B (Mi2) starts operating. Here, as an example, it is assumed that the subsequent module B (Mi2) processes image data faster than the preceding module A (Mi1).

[0031] That is, the solid line 402a has a steeper gradient than the solid line 401a. Let "d" be the difference between the LE counter value of the preceding module A (Mi1) and the LE counter value of the subsequent module B (Mi2) at the same time. At time t1, the difference d between the LE counter values of the preceding module A (Mi1) and the subsequent module B (Mi2) is d=da1. The i-th sequencer 101-i monitors the difference d of the LE counter values. When d=0, the processing of the subsequent module B (Mi2) catches up with the processing of the preceding module A (Mi1); when d<0, the processing of the subsequent module B (Mi2) has overtaken the processing of the preceding module A (Mi1).

[0032] At time t2, the preceding module A (Mi1) has not yet completed processing for one frame. Assume that at time t2, the difference d between the LE counter values of the preceding module A (Mi1) and the subsequent module B (Mi2) becomes d=da2 (<da1). If the processing of the subsequent module B (Mi2) continues at this processing speed, the processing of the subsequent module B (Mi2) will catch up with the processing of the preceding module A (Mi1) before the preceding module A (Mi1) completes the processing of one frame.

[0033] Therefore, if the difference da2 of the LE counter values at time t2 is equal to or less than the predetermined threshold value tha (≧0), the i-th sequencer 101-i outputs a FLLW_NOGO signal to the subsequent module B (Mi2), and the processing of the subsequent module B (Mi2) is suspended. The generation period of the FLLW_NOGO signal is from time t1 to time t4.

[0034] Assuming that the FLLW_NOGO signal falls at time t3, the subsequent module B (Mi2) resumes processing. At time t4, the preceding module A (Mi1) completes the processing of one frame. The preceding module A (Mi1) and the subsequent module B (Mi2) execute the processing shown from time t0 to t4 in frame units.

[0035] Thus, when the difference d in the LE counter values ​​narrows to a predetermined threshold tha (referred to as the "Too Close state"), the i-th sequencer 101-i controls the subsequent module B (Mi2) to temporarily stop, so the processing of the subsequent module B (Mi2) will not be able to keep up with the processing of the preceding module A (Mi1).

[0036] This prevents the preceding module A (Mi1) and the subsequent module B (Mi2) from accessing data at the same address at the same time, and also prevents the subsequent module B (Mi2) from accessing data before it has been processed by the preceding module A (Mi1) when the subsequent module B (Mi2) performs data processing using data already processed by the preceding module A (Mi1).

[0037] <Buffer overflow prevention control> Next, buffer overflow prevention control will be explained. Buffer overflow prevention control is a control mechanism in which the i-th sequencer 101-i prevents buffer overflow caused by an increase in the amount of data written to a buffer such as DRAM 104 or SRAM 152 by the preceding module A (Mi1).

[0038] Specifically, for example, buffer overflow prevention control means that the i-th sequencer 101-i controls the difference d between the number of lines of image data processed by the preceding module A (Mi1) and the number of lines of image data processed by the subsequent module B (Mi2) so that it does not exceed a predetermined threshold thb.

[0039] Figure 5 is an explanatory diagram showing an example of buffer overflow prevention control. The upper part of Figure 5 is a graph 500 showing the temporal changes in data processing of the preceding module A (Mi1) and the subsequent module B (Mi2), and the lower part is a timing chart 510 showing the control by the i-th sequencer 101-i.

[0040] In Graph 500, the horizontal axis represents time, and the vertical axis represents the LE counter value counted by the i-th sequencer 101-i. In Graph 500, the solid line 401b shows the temporal change of the actual LE counter value in the preceding module A (Mi1), where the LE counter value increases monotonically over time. The dotted line 410b shows the temporal change of the ideal LE counter value in the preceding module A (Mi1), where the LE counter value increases monotonically over time. The solid line 402b shows the temporal change of the actual LE counter value in the subsequent module B (Mi2), where the LE counter value increases monotonically over time.

[0041] Assume that at time t0, the i-th sequencer 101-i outputs a LEAD_KICKVD signal to the pre-stage module A(Mi1), and the pre-stage module A(Mi1) starts up. The pre-stage module A(Mi1) processes the image data as shown by solid line 401b.

[0042] At time t1, the i-th sequencer 101-i outputs the FLLW_KICKVD signal to the subsequent module B (Mi2), and the subsequent module B (Mi2) starts operating. Here, as an example, it is assumed that the preceding module A (Mi1) processes image data faster than the subsequent module B (Mi2).

[0043] In other words, the solid line 401b has a steeper gradient than the solid line 402b. At time t1, the difference d between the LE counter values ​​of the preceding module A (Mi1) and the succeeding module B (Mi2) is d = db1. The i-th sequencer 101-i monitors the difference d between the LE counter values. If d ≥ thb, the processing of the preceding module A (Mi1) will pull ahead of the processing of the succeeding module B (Mi2), causing a buffer overflow by the preceding module A (Mi1).

[0044] At time t2, the preceding module A (Mi1) has not yet completed processing for one frame. At time t2, the difference d between the LE counter values ​​of the preceding module A (Mi1) and the succeeding module B (Mi2) is d = db2 (> db1). If the processing of the preceding module A (Mi1) continues at this processing speed, a buffer overflow will occur due to the processing of the preceding module A (Mi1).

[0045] Therefore, if the difference db2 of the LE counter values ​​at time t2 is greater than or equal to a predetermined threshold thb, the i-th sequencer 101-i outputs a LEAD_NOGO signal to the preceding module A(Mi1), and the processing of the preceding module A(Mi1) is temporarily suspended. The period during which the LEAD_NOGO signal is generated is from time t1 to time t4.

[0046] If the LEAD_NOGO signal falls at time t3, the preceding module A (Mi1) resumes processing. At time t4, the succeeding module B (Mi2) completes processing for one frame. The preceding module A (Mi1) and the succeeding module B (Mi2) execute the processing shown from time t0 to t4 on a frame-by-frame basis.

[0047] Thus, when the difference d in the LE counter values ​​widens to a predetermined threshold thb (referred to as the "Too Far state"), the i-th sequencer 101-i controls the preceding module A (Mi1) to temporarily stop, so that the processing of the subsequent module B (Mi2) is not separated from the processing of the preceding module A (Mi1). This suppresses buffer overflow caused by the preceding module A (Mi1).

[0048] <How to start the Mi module group> Next, we will explain how to start the module group Mi using the i-th sequencer 101-i. There are two methods for starting the module group Mi: one is host control (Figure 6), and the other is KICKVD control (Figure 7).

[0049] Figure 6 is an explanatory diagram illustrating an example of a host-controlled startup method for the module group Mi. Host control is a startup method in which the processor 106 starts only the first module Mi1 of the module group Mi, and the subsequent modules Mi2, Mi3, ... are controlled by the i-th sequencer 101-i, which controls the start and end of processing of the subsequent modules based on the start and end of processing of the preceding modules, without involving the processor 106. In Figure 6, this is explained using j=3.

[0050] The processor 106 outputs an ACT signal to the front-end module A (Mi1) of the module group Mi, activating the front-end module A (Mi1). Upon activation, the front-end module A (Mi1) outputs a VD signal to the i-th sequencer 101-i, sequentially reads image data for one frame from the first line, and writes the processing results to the buffer (DRAM 104 or SRAM 152) 600.

[0051] When the i-th sequencer 101-i has elapsed a predetermined period since receiving the VD signal, it outputs the FLLW_KICKVD signal to the middle module B(Mi2), activating the middle module B(Mi2). Upon activation, the middle module B(Mi2) outputs the VD signal to the i-th sequencer 101-i, reads one frame's worth of image data sequentially from the first line of the buffer 600, and writes the processing results to the buffer 600.

[0052] When the i-th sequencer 101-i has elapsed a predetermined period since the input of the VD signal, it outputs the FLLW_KICKVD signal to the final stage module C(Mi3), thereby activating the final stage module C(Mi3). Upon activation, the final stage module C(Mi3) reads image data for one frame sequentially from the buffer 600 starting from the first line and outputs image data for one frame as the processing result.

[0053] In this way, under host control, the processor 106 only needs to output an ACT signal to the front-end module A (Mi1), and from there the i-th sequencer 101-i can control the module group Mi without going through the processor 106.

[0054] Figure 7 is an explanatory diagram illustrating an example of how to start the module group Mi using KICKVD control. KICKVD control is a startup method in which the processor 106 outputs a KICKVD signal to the i-th sequencer 101-i to start the i-th sequencer 101-i, and thereafter, without going through the processor 106, the i-th sequencer 101-i controls the start and end of processing for modules Mi1, Mi2, Mi3, etc., based on the start and end of processing of the preceding modules. In Figure 7, this is explained using j=3.

[0055] For example, suppose the image sensor equipped with the data processing device 100 is set to shooting mode when the release button is pressed. Upon this press, the processor 106 outputs a KICKVD signal to the i-th sequencer 101-i, which then starts the i-th sequencer 101-i.

[0056] The i-th sequencer 101-i outputs a KICKVD signal to the front-end module A (Mi1) of the module group Mi, thereby activating the front-end module A (Mi1). Upon activation, the front-end module A (Mi1) outputs a VD signal to the i-th sequencer 101-i, sequentially reads one frame's worth of image data from the first line, and writes the processing results to the buffer (DRAM 104 or SRAM 152) 600.

[0057] When the i-th sequencer 101-i has elapsed a predetermined period since receiving the VD signal, it outputs the FLLW_KICKVD signal to the middle module B(Mi2), activating the middle module B(Mi2). Upon activation, the middle module B(Mi2) outputs the VD signal to the i-th sequencer 101-i, reads one frame's worth of image data sequentially from the first line of the buffer 600, and writes the processing results to the buffer 600.

[0058] When the i-th sequencer 101-i has elapsed a predetermined period since the input of the VD signal, it outputs the FLLW_KICKVD signal to the final stage module C(Mi3), thereby activating the final stage module C(Mi3). Upon activation, the final stage module C(Mi3) reads image data for one frame sequentially from the buffer 600 starting from the first line and outputs image data for one frame as the processing result.

[0059] In this way, with KICKVD control, the i-th sequencer 101-i can control the module group Mi without going through the processor 106.

[0060] <Coordination between the preceding and succeeding modules> Next, we will explain the coordinated operation between the preceding module and the succeeding module.

[0061] Figure 8 is an explanatory diagram illustrating an example of the coordinated operation between one pre-stage module A (Mi1) and two post-stage modules B (Mi2) and C (Mi3). Here, the combination of pre-stage module A (Mi1) and post-stage module B (Mi2) is denoted as pair_0, and the combination of pre-stage module A (Mi1) and post-stage module C (Mi3) is denoted as pair_1.

[0062] Since the preceding module A (Mi1) is a module common to both pair_0 and pair_1, it writes data to buffer 600 during the processing of each of pair_0 and pair_1.

[0063] The subsequent module B (Mi2) reads the data written to buffer 600 by the preceding module A (Mi1) from buffer 600 and executes the processing of the subsequent module B (Mi2). The subsequent module C (Mi3) reads the data written to buffer 600 by the preceding module A (Mi1) from buffer 600 and executes the processing of the subsequent module C (Mi3).

[0064] For pair_0, sequencer i 101-i outputs the FLLW_KICKVD signal to the subsequent module B (Mi2) when activating it, and for pair_1, it outputs the FLLW_KICKVD signal to the subsequent module C (Mi3) when activating it.

[0065] When the i-th sequencer 101-i performs buffer overflow control (Figure 5) for pair_0 and pair_1, it outputs a LEAD_NOGO signal to the preceding module A (Mi1).

[0066] When the i-th sequencer 101-i performs overtaking prevention control (Figure 4) for pair_0, it outputs the FLLW_NOGO signal to the next module B (Mi2), and when it performs overtaking prevention control (Figure 4) for pair_1, it outputs the FLLW_NOGO signal to the next module C (Mi3).

[0067] Furthermore, when the i-th sequencer 101-i prioritizes QoS (Quality of Service) control for pair_0, it outputs the FLLW_QOS signal to the subsequent module B (Mi2), and when prioritizing QoS control for pair_1, it outputs the FLLW_QOS signal to the subsequent module C (Mi3).

[0068] QoS priority control is a control method that adjusts the priority of the target module, increasing the number of buffer accesses the higher the priority. For example, in pair_0, if the processing of the subsequent module B (Mi2) is delayed and the difference between it and the processing of the preceding module A (Mi1) widens, if the i-th sequencer 101-i outputs a LEAD_NOGO signal to the preceding module A (Mi1), the difference between the processing of the preceding module A (Mi1) and the processing of the subsequent module B (Mi2) will narrow, but the difference between the processing of the preceding module A (Mi1) and the processing of the subsequent module C (Mi3) will also narrow.

[0069] If the FLLW_NOGO signal is output to the subsequent module C (Mi3), the difference between the processing of the preceding module A (Mi1) and the processing of the subsequent module C (Mi3) will widen, but the processing of pair_0 as a whole will be delayed. In such a case, the i-th sequencer 101-i outputs the FLLW_QOS signal to the subsequent module B (Mi2) to give module B (Mi2) a higher priority than module C (Mi3). Alternatively, the i-th sequencer 101-i outputs the FLLW_QOS signal to module C (Mi3) to give module C (Mi3) a lower priority than module B (Mi2). That is, the number of times module B (Mi2) accesses buffer 600 will be greater than the number of times module C (Mi3) accesses buffer 600.

[0070] In this way, QoS priority control enables control that prevents a decrease in overall processing speed, which cannot be addressed by overtaking prevention control or buffer overflow control. Note that although Figure 8 shows two downstream modules, three or more may be used.

[0071] Figure 9 is an explanatory diagram illustrating an example of the coordinated operation between two pre-stage modules A(Mi1) and B(Mi2) and one post-stage module C(Mi3). Here, the combination of pre-stage module A(Mi1) and post-stage module C(Mi3) (channel ch0 side) is referred to as pair_2, and the combination of pre-stage module B(Mi2) and post-stage module C(Mi3) (channel ch1 side) is referred to as pair_3.

[0072] In the processing of pair_2, the preceding module A (Mi1) writes data to buffer 600, and in the processing of pair_3, the preceding module B (Mi2) writes data to buffer 600.

[0073] In the processing of pair_2, the subsequent module C (Mi3) reads the data written to buffer 600 by the preceding module A (Mi1) from buffer 600, receives it on channel ch0, and executes the processing of the subsequent module C (Mi3). In the processing of pair_3, the subsequent module C (Mi3) reads the data written to buffer 600 by the preceding module B (Mi2) from buffer 600, receives it on channel ch1, and executes the processing of the subsequent module C (Mi3).

[0074] For the i-th sequencer 101-i, when activating the subsequent module C(Mi3) for pair_2, it outputs the FLLW_KICKVD signal to channel ch0 of the subsequent module B(Mi2), and for the i-th sequencer 101-i, when activating the subsequent module C(Mi3) for pair_1, it outputs the FLLW_KICKVD signal to channel ch1 of the subsequent module C(Mi3).

[0075] When the i-th sequencer 101-i performs buffer overflow control (Figure 5) on pair_2, it outputs a LEAD_NOGO signal to the preceding module A (Mi1), and when performing buffer overflow control (Figure 5) on pair_3, it outputs a LEAD_NOGO signal to the subsequent module B (Mi2).

[0076] When the i-th sequencer 101-i performs overtaking prevention control (Figure 4) for pair_2, it outputs the FLLW_NOGO signal to channel ch0 of the subsequent module C (Mi3), and when performing overtaking prevention control (Figure 4) for pair_3, it outputs the FLLW_NOGO signal to channel ch1 of the subsequent module C (Mi3).

[0077] With this control, for example, if pair_2 is in a Too Far state and pair_3 is in a Too Close state, the preceding module A (Mi1) is paused by the LEAD_NOGO signal, the following module C (Mi3) is paused by the FLLW_NOGO signal, and the preceding module B (Mi2) operates without being paused.

[0078] Furthermore, when the i-th sequencer 101-i performs QoS priority control for pair_2, it outputs the FLLW_QOS signal to channel ch0 of the downstream module C(Mi3), and when performing QoS priority control for pair_3, it outputs it to channel ch1 of the downstream module C(Mi3). Thus, QoS priority control is also possible in Figure 9. Note that although the number of upstream modules is set to 2 in Figure 9, it may be 3 or more.

[0079] <Virtual line counter tracking type QoS priority control> Figure 10 is a graph showing an example of virtual line counter tracking type QoS priority control. Virtual line counter tracking type QoS priority control is a process in which, when there is only one module Mij in module group Mi (j=1), the i-th sequencer 101-i prioritizes QoS control of module Mi1 by tracking the line counter value of a virtual preceding module.

[0080] In graph 1000, the horizontal axis represents time, and the vertical axis represents the LE counter value counted by the i-th sequencer 101-i. The solid line 1001 represents the temporal change in the line counter value by a virtual pre-stage module. The i-th sequencer 101-i counts the line counter value by a virtual pre-stage module and resets it at time t1.

[0081] The solid lines 1002, 1021, and 1022 represent the temporal changes in line counter values ​​by module Mij. The dashed line 1003 represents the temporal changes in line counter values ​​that define the threshold range for the i-th sequencer 101-i to output the FLLW_QOS signal to module Mi1. This threshold range is defined by the dotted lines 1003a and 1003b, which are equidistant from the solid line 1003.

[0082] As shown by solid line 1002, if the line counter value by module Mij is within the threshold range defined by dotted lines 1003a and 1003b, the i-th sequencer 101-i does not perform QoS priority control for module Mi1.

[0083] As shown by the solid line 1021, if the line counter value of module Mij exceeds the line counter value of dotted line 1003a at time t2, the i-th sequencer 101-i outputs an FLLW_QOS signal to module Mi1, setting its priority lower than its current priority. This reduces the number of times module Mi1 accesses buffer 600, causing the slope of the solid line 1021 to decrease at time t3.

[0084] Furthermore, as shown by the solid line 1022, if the line counter value of module Mij falls below the line counter value of dotted line 1003b at time t4, the i-th sequencer 101-i outputs an FLLW_QOS signal to module Mi1, setting it to a higher priority than its current priority. This increases the number of times module Mi1 accesses buffer 600, causing the slope of the solid line 1022 to increase at time t5.

[0085] Thus, even with the standalone Mi1 module, the buffer access speed can be adjusted using virtual line counter-following QoS priority control.

[0086] <Area setting table> Figure 11 is an explanatory diagram showing an example of a region setting table 153. The region setting table 153 is stored in a memory (not shown) that is different from the SRAM 152 in the SRAM system 105. The region setting table 153 has a module group ID 1101, a region ID 1102, and an SRAM physical start address 1103.

[0087] Module group ID 1101 is identification information that uniquely identifies module group Mi. The data from module group Mi sent via interconnect 103 includes module group ID 1101, and the conversion unit 151 identifies an entry in the area setting table 153 from the module group ID 1101 included in the acquired data.

[0088] Region ID 1102 is identification information that uniquely identifies regions A1 to An of the physical address space PS of SRAM152 used by module group Mi. The physical address space PS is divided into regions A1 to An. When regions A1 to An are not distinguished, they are simply referred to as region Ai. SRAM physical start address 1103 is the start address of region Ai, which is identified by region ID 1102.

[0089] The SRAM physical start address 1103 is rewritable by the processor 106. Specifically, for example, in an imaging device equipped with a data processing device 100, when the shooting mode is switched, it is updated to a value corresponding to the new shooting mode.

[0090] <Address space remapping> Figure 12 is an explanatory diagram showing an example of address space remapping. Module group Mi specifies the data access destination using an address in the logical address space LS. The interconnect 103 determines whether the data from module group Mi is accessed from DRAM 104 or SRAM 152.

[0091] Specifically, for example, if the access destination of data specified by module group Mi is within the first logical address area LS1 of the logical address space LS, the interconnect 103 determines that the access destination is SRAM 152, and if it is within the second logical address area LS2 of the logical address space LS, the interconnect 103 determines that the access destination is DRAM 104. For example, if the access destination address ladr2 of data from module group M2 is within the first logical address area LS1, it is stored in SRAM 152, and if it is within the second logical address area LS2, it is stored in DRAM 104.

[0092] The conversion unit 151 obtains the module group ID 1101 contained in the data from module group Mi if the access destination is within the first logical address area LS1. If the data is read from buffer 600, it contains the module group ID 1101 of module group Mi and the address in the logical address space of the read destination. If the data is written to buffer 600, it contains the data to be written, the module group ID 1101 of module group Mi, and the address in the logical address space LS of the write destination.

[0093] Next, the conversion unit 151 identifies an entry in the region setting table 153 that includes the acquired module group ID 1101, and obtains the SRAM physical start address 1103 of the identified entry.

[0094] Then, the conversion unit 151 substitutes the acquired SRAM physical start address 1103 into the following equation (1) to calculate the remapped SRAM address.

[0095] Sadri=ladri / (r_STADR(i+1)-r_STADRi)+r_STADRi···(1)

[0096] Sadri is the remapped SRAM address accessed by module group Mi. ladri is the address in the logical address space LS that module group Mi accesses. r_STADRi is the SRAM physical start address 1103 obtained by the translation unit 151 from the region setting table 153. r_STADR(i+1) is the SRAM physical start address 1103 of region A(i+1), which is one level above region Ai of the SRAM physical start address 1103 obtained by the translation unit 151 from the region setting table 153.

[0097] According to equation (1) above, a ring buffer can be constructed for each region Ai within the physical address space PS of SRAM152. This allows SRAM152 to be shared by multiple module groups M1 to Mn, and the memory capacity can be reduced compared to implementing SRAM152 for each module group Mi.

[0098] Thus, according to this embodiment, the i-th sequencer 101-i centrally manages the start, progress, and end of data processing for the module group Mi. Therefore, even if there are fluctuations in the processing speed of the module group Mi or an increase in the delay amount of the interconnect 103, the difference in LE counter values ​​is monitored, which suppresses overtaking of preceding modules by subsequent modules and buffer overflows by preceding modules.

[0099] Furthermore, according to this embodiment, the conversion unit 151 virtually uses the region Ai in the physical address space PS of the SRAM 152 as a ring buffer, and remaps the addresses in the logical address space LS specified by the module group Mi to the addresses in the link buffer. This makes it possible to reduce the size of region Ai. In addition, since a ring buffer for region Ai is set for each module group Mi, it can be shared using a single SRAM 152.

[0100] Based on the above, this embodiment makes it possible to improve the efficiency of data processing by the module. [Explanation of Symbols]

[0101] 100 Data processing unit, 101 Sequencer set, 101-i i-th Sequencer, 102 Module set, 103 Interconnect, 105 SRAM system, 106 Processor, 151 Conversion unit, 152 SRAM, 153 Area setting table, 1101 Module group ID, 1102 Area ID, LS Logical address space, PS Physical address space, Mi Module group

Claims

1. A first module that performs first data processing on the first data, A second module that performs a second data processing on the first data after the start of the first data processing, The system includes a first sequencer that controls either the first module or the second module based on the difference between a first position where the first data is processed by the first data processing and a second position where the first data is processed by the second data processing, The first sequencer is a data processing device that controls the first module to temporarily suspend the first data processing when the difference exceeds a first threshold.

2. A data processing device according to claim 1, The first sequencer is a data processing device that controls the second module to temporarily suspend the second data processing when the difference falls below a second threshold.

3. A data processing device according to claim 1 or claim 2, A data processing device which controls the first sequencer to start the first module, and after detecting the start of the first module, controls the second module to start.

4. A data processing device according to any one of claims 1 to 3, The first sequencer is a data processing device that controls how often the first module accesses the first data based on the first position.

5. A data processing device according to claim 4, The first sequencer is a data processing device that reduces the frequency with which the first module accesses the first data when the first position exceeds the upper limit of a predetermined range.

6. A data processing device according to claim 5, The first sequencer is a data processing device that increases the frequency with which the first module accesses the first data when the first position exceeds the lower limit of a predetermined range.

7. A data processing device according to any one of claims 4 to 6, The first sequencer is a data processing device that controls how often the second module accesses the first data based on the second position.

8. A data processing device according to claim 7, The first sequencer is a data processing device that controls the frequency with which the first module and the second module access the first data based on the first position and the second position.

9. A data processing device according to any one of claims 1 to 8, The first data is image data, and the data processing device is also used.

10. A data processing device according to any one of claims 1 to 9, A data processing device comprising a third module that performs a third data processing on the first data after the start of the second data processing.

11. A data processing device according to claim 10, The first sequencer is a data processing device that controls the second module to temporarily suspend the second data processing when the difference between the second position and the third position where the first data is processed by the third data processing becomes greater than or equal to a third threshold.

12. A data processing device according to claim 11, The first sequencer is a data processing device that controls the third module to temporarily suspend the third data processing when the difference between the second position and the third position falls below a fourth threshold.

13. A data processing device according to claim 12, The first sequencer is a data processing device that controls the third module to temporarily suspend the third data processing when the difference between the second position and the third position where the first data is processed by the third data processing falls below a fourth threshold.

14. A data processing device according to claim 13, A memory unit having a first memory accessed by the first module and the second module, A control unit that controls the capacity of the first memory and A data processing device equipped with the following features.

15. A data processing device according to claim 14, The control unit is a data processing device that controls the capacity of the first memory according to the type of data acquired.

16. An imaging device comprising the data processing device described in any one of claims 1 to 15.

Images