Signal processing device and signal processing method

Abstract

The present invention provides a signal processing device that can process signals to be processed even if there is a delay in signal transfer from any of the multiple storage units. [Solution] The signal processing device (1) includes a read unit (13-1 to 13-3) that reads signals from frame memories (11-1, 11-2), line memories (14-1, 14-2) that store line-unit signals, a control unit (15) that stops reading signals corresponding to the next period if the reading of line-unit signals from at least one frame memory is not completed within the period corresponding to that line unit, and starts reading signals corresponding to the next period after the reading of the unread signals is completed, and a processing unit (16) that processes line-unit signals of corresponding periods.

Description

Technical Field

[0001] The present disclosure relates to a signal processing apparatus and a signal processing method.

Background Art

[0002] As a signal transfer means in a digital signal processing system, AXI (Advanced eXtensible IntErface) is known. For example, Patent Document 1 describes an image processing apparatus having an AXI bus. The AXI bus is a bus interface having a handshake function and is widely used in large-scale digital signal processing systems. The handshake function is a function of transmitting the next signal after confirming that the other party has received the signal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When processing signals to be processed read from a plurality of storage units in a fixed processing unit, if the reading of a signal from any of the storage units is delayed, there is a problem that the processing of the signals to be processed cannot be performed. For example, when comparing pixels at the same coordinates of video signals read in line units from two frame memories in which video signals are stored in units of one frame via an AXI bus, the access to the AXI bus may be concentrated so that the reading of a signal from one of the frame memories does not complete within the reading period corresponding to the line unit. In this case, the comparison operation of pixels at the same coordinates cannot be performed, which causes video disturbance in line units.

[0005] This disclosure aims to solve the above-mentioned problems and to provide a signal processing device that can process signals even if there is a delay in reading signals from any of the multiple storage units. [Means for solving the problem]

[0006] The signal processing device according to this disclosure includes: a reading unit that reads signals stored in a plurality of first storage units at periodic intervals corresponding to a certain processing unit and stores the signals of the processing units read from the plurality of first storage units in a plurality of second storage units; a control unit that, if the reading of a signal of a processing unit from at least one of the plurality of first storage units is not completed within the period corresponding to the processing unit, controls the reading unit to stop reading the signals corresponding to the next period, and starts reading the signals corresponding to the next period after the reading of the signals that were not completed is completed; and a processing unit that processes the signals of processing units of corresponding periods stored in the plurality of second storage units. [Effects of the Invention]

[0007] According to this disclosure, if the reading of a processing unit signal from at least one first storage unit is not completed within the period corresponding to that processing unit, the reading unit is controlled to stop reading signals of the next period corresponding to the processing unit from multiple first storage units. After the reading of the signals that were not completed is completed, the reading of signals of the next period corresponding to the processing unit is started, and the signals of the processing units for the corresponding period read from multiple second storage units are processed together. As a result, the signal processing device according to this disclosure can process the signals to be processed even if there is a delay in reading signals from any of the multiple storage units. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example configuration of a signal processing device according to Embodiment 1. [Figure 2] This is a conceptual diagram illustrating the overview of the overlay process. [Figure 3] This is a conceptual diagram illustrating the overlay processing that occurs when reading is not completed within the time period corresponding to one line. [Figure 4] This is a flowchart showing the signal processing method according to Embodiment 1. [Figure 5] This figure shows an example of the video display timing for one frame of a video signal. [Figure 6] This is a conceptual diagram illustrating the process in Embodiment 1, which outputs the result of a comparison operation between pixels at the same coordinates of video signals read from two frame memories as a video signal. [Figure 7] This is a timing chart showing each signal during the normal operation of the signal processing device according to Embodiment 1. [Figure 8] This is a timing chart illustrating the overview of pixel-level comparison calculations. [Figure 9] This is a timing chart showing the signals of the signal processing device according to Embodiment 1 in the case where reading is not completed within the reading period for one line. [Figure 10] This is a timing chart showing the signals from the output line memory and switching unit during the normal operation of the signal processing device according to Embodiment 1. [Figure 11] This is a timing chart showing the signals from the output line memory and switching unit of the signal processing device according to Embodiment 1 in the case where reading is not completed within the reading period for one line. [Figure 12] This is a block diagram showing an example configuration of a signal processing device according to Embodiment 2. [Figure 13] This is a timing chart showing each signal from the output frame memory during the normal operation of the signal processing device according to Embodiment 2. [Figure 14] This is a timing chart showing each signal from the output frame memory of the signal processing device according to Embodiment 2 in the case where reading is not completed within the reading period for one line. [Figure 15]This is a timing chart showing the signals from the output frame memory of the signal processing device according to Embodiment 2 in the case where the reading of two lines has not been completed. [Figure 16] This is a conceptual diagram illustrating a modified version of the process in Embodiments 1 and 2, which outputs the result of a comparison operation between pixels that are not at the same coordinates in video signals read from two frame memories as a video signal. [Modes for carrying out the invention]

[0009] Embodiment 1. (Overview of signal processing equipment) Figure 1 is a block diagram showing an example configuration of the signal processing device 1 according to Embodiment 1. In Figure 1, the signal processing device 1 transfers video signals line by line from two frame memories 11-1 and 11-2 via the AXI bus 3, performs a comparison operation on pixels at the same coordinates within one frame of each video signal, and outputs the processing result. Various forms of comparison operations are possible, for example, overlay processing. A line is one of the processing units on which comparison operations are performed.

[0010] Figure 2 is a conceptual diagram illustrating the overview of the overlay process. As shown in Figure 2, the overlay process is the process of generating an overlay image C by superimposing another image B on top of one image A. For example, image A is the image of the video signal stored in frame memory 11-1, and image B is the image of the video signal stored in frame memory 11-2. The signal processing device 1 treats the areas where the color is black in each frame image of image B as transparent and superimposes them on image A. Specifically, it determines whether each pixel in each frame image of image B is black or not, converts black pixels to transparent pixels, and outputs an image signal superimposed on the pixels at the same coordinates in each frame image of image A.

[0011] If the reading of the line - unit signal from one of the frame memories 11 - 1 and 11 - 2 is not completed within the period corresponding to this line unit, the signals of the corresponding line units cannot be processed. Here, the reading of the line - unit signal from the frame memory corresponds to the transfer time of the signal on the AXI bus 3 in the period corresponding to this line unit.

[0012] Figure 3 is a conceptual diagram showing the outline of the overlay process when the reading within the period corresponding to one line unit is not completed. For example, when reading the video signal of video A1 from the frame memory 11 - 1 line - by - line, if the reading of the signal corresponding to line D is not completed within the period corresponding to this line D, the signal of the next line is read. Therefore, as shown in Figure 3, the part of video A1 in line D is in a disordered state. For each pixel of each frame image of video B, it is determined whether it is black or not, and the black pixels are converted into transparent pixels. Next, a video C1 in which video B is overlaid on the pixels of the same coordinates on each frame image of video A1 is output. As a result, video C1 has video disorder in line D.

[0013] The factors that the reading is not completed within the period corresponding to one line unit may be considered as follows. For example, there is an access concentration to the AXI bus 3. The AXI bus 3 has a handshake function, and it takes time for the transmission - side and the reception - side to exchange signals until communication is established for each access. When multiple accesses to the frame memories from the reading part via the AXI bus 3 are performed simultaneously, this time becomes a factor of delay. Also, there is an influence such as access from the video - signal input parts 2 - 1 to 2 - 3 to the frame memories 11 - 1 and 11 - 2 via the AXI bus 3. The video - signal input parts 2 - 1 to 2 - 3 input video signals from arbitrary external devices, and delays also occur due to poor contact or plug - in and unplug of the video - input cable.

[0014] Generally, during the transmission of a single frame of video signal, there are horizontal blanking periods and vertical blanking periods during which the image of the video signal is not displayed on the screen. The horizontal blanking period is the time required to prepare the screen after one line of image data has been drawn and before the next line is drawn. The vertical blanking period is the time required to prepare the screen after one frame of image data has been drawn and before the next frame is drawn.

[0015] In the first embodiment, the signal processing device 1 stops reading line-by-line signals from the remaining frame memories at the horizontal timing if, for example, the reading of a line-by-line signal from any of the multiple frame memories is delayed for a period longer than the period corresponding to one line. Next, the signal processing device 1, in synchronization with the frame memory where the signal reading was delayed, starts reading the signal from the line following the delayed line at the next horizontal timing once the reading of the signal from the delayed line is completed. By processing (comparing or calculating) the signals of lines whose readout was delayed during the horizontal blanking period and the vertical blanking period, and outputting the processing results, it is possible to output the processing results for pixels at the same coordinates for all pixels within a single frame period.

[0016] Furthermore, the signal processing device 1 includes an output line memory 17 after the processing unit 16, and a switching unit 18 that switches between the processing unit 16 and the output line memory 17. By switching between outputting the normal processing result and outputting the processing result when a delay occurs, the processing result for each line can be output continuously. As a result, the image distortion shown in Figure 3 does not occur, and it is possible to display a normal image without any distortion, such as in Figure 2, where not a single pixel or line is distorted.

[0017] (Example of signal processing device configuration) As shown in Figure 1, the signal processing device 1 is a device connected to input units 2-1 to 2-3 via the AXI bus 3, and includes frame memories 11-1 to 11-3, a synchronization signal generation unit 12, readout units 13-1 to 13-3, line memories 14-1 and 14-2, a control unit 15, a processing unit 16, an output line memory 17, and a switching unit 18.

[0018] Input units 2-1 to 2-3 receive video signals from external devices, which are provided separately from the signal processing device 1, via the AXI bus 3. Input unit 2-1 stores the input video signal via the AXI bus 3 into frame memory 11-1. Similarly, input unit 2-2 stores the input video signal via the AXI bus 3 into frame memory 11-2. Input unit 2-3 stores the input video signal via the AXI bus 3 into frame memory 11-3.

[0019] AXI bus 3 is a bus with a handshake function. In the example shown in Figure 1, AXI bus 3 connects the input units 2-1 to 2-3 to the frame memories 11-1 to 11-3, and the frame memories 11-1 to 11-3 to the read units 13-1 to 13-3. That is, signals are transferred between the input units 2-1 to 2-3 and the frame memories 11-1 to 11-3 via AXI bus 3, and signals are also transferred between the frame memories 11-1 to 11-3 and the read units 13-1 to 13-3 via AXI bus 3.

[0020] Frame memories 11-1 to 11-3 are first storage units that store video signals transferred from input units 2-1 to 2-3 via the AXI bus 3 on a frame-by-frame basis. The video signals stored in frame memories 11-1 to 11-3 on a frame-by-frame basis are read out line by line by read units 13-1 to 13-3 via the AXI bus 3. In the configuration example shown in Figure 1, the signal processing device 1 is equipped with frame memories 11-1 to 11-3, but the frame memories may be provided by an external device separate from the signal processing device 1. In this case, the frame memories provided by the external device are connected to the readout units 13-1 to 13-3 via the AXI bus 3.

[0021] The synchronization signal generation unit 12 generates synchronization signals. For example, the synchronization signal generation unit 12 generates a dot clock signal (hereinafter referred to as "dot clock") which serves as a reference for displaying the video signal, a vertical synchronization signal (hereinafter referred to as "Vsync"), a horizontal synchronization signal (hereinafter referred to as "Hsync"), and a valid video signal (hereinafter referred to as "Valid"). The dot clock is a clock signal that determines when a display acquires and draws data for each pixel. The dot clock defines the number of pixels a display processes per second and serves as the basis for overall video signal processing. Vsync is a synchronization signal that notifies the display to begin drawing the next frame after the drawing of one frame, corresponding to the entire screen, is complete. This signal occurs after all lines have been drawn from top to bottom of the screen and indicates when the next frame will be drawn. Hsync is a synchronization signal used to notify the display that it is ready to start drawing the next line after the previous line (a horizontal row of pixels) has been drawn. This signal is generated when each line has finished drawing and determines when to start drawing the next line. The active video signal is the signal that indicates the portion of the image that will actually be displayed on the screen. It is the pixel data itself that will be displayed on the screen, and it contains information such as what color and brightness each pixel should be displayed with. During the blanking period, signals are not displayed, and only this active video signal is shown on the screen.

[0022] The reading units 13-1 to 13-3 read the signals stored in the frame memories 11-1 to 11-3 at periodic intervals corresponding to a line unit, which is a fixed processing unit. The read unit 13-1 reads the video signal stored in the frame memory 11-1 line by line at periodic intervals corresponding to the line, and stores the read line-by-line signal (write data (1) in Figure 1) in the line memory 14-1 via the AXI bus 3. The read unit 13-2 reads the video signal stored in the frame memory 11-2 line by line at periodic intervals corresponding to the line, and stores the read line-by-line signal (write data (2) in Figure 1) in the line memory 14-2 via the AXI bus 3. Furthermore, the readout unit 13-3 reads the video signals stored in the frame memory 11-3 line by line at periodic intervals corresponding to those periods, and outputs the readout line-by-line signals (video signals (2) in Figure 1) to the outside of the device via the AXI bus 3.

[0023] Line memories 14-1 and 14-2 are second storage units that store line-by-line signals read from frame memories 11-1 and 11-2. For example, line memory 14-1 stores two lines of line-by-line video signals (write data (1)) read from frame memory 11-1 by read unit 13-1. Line memory 14-2 stores two lines of line-by-line video signals (write data (2)) read from frame memory 11-2 by read unit 13-2. In the configuration example shown in Figure 1, the signal processing device 1 is equipped with line memories 14-1 and 14-2, but the line memories may be provided by an external device separate from the signal processing device 1. In this case, the line memories provided by the external device are connected to the read units 13-1 and 13-2 and the processing unit 16 provided by the signal processing device 1.

[0024] The control unit 15 controls the reading of signals from frame memories 11-1 and 11-2 by the reading units 13-1 and 13-2, the writing and reading of signals to line memories 14-1 and 14-2, the writing of signals to output line memory 17, the reading of signals from output line memory 17, and the switching by the switching unit 18, based on Vsync, Hsync, and Valid generated by the synchronization signal generation unit 12. For example, if the readout of a line-by-line signal from at least one of the frame memories 11-1, 11-2 is not completed within the period corresponding to that line, the control unit 13-1, 13-2 controls the readout units 13-1, 13-2 to stop the readout of the signal corresponding to the next period, and then starts the readout of the signal corresponding to the next period after the readout of the uncompleted signal is completed.

[0025] The processing unit 16 processes line-level signals for corresponding periods stored in line memories 14-1 and 14-2. For example, the processing unit 16 performs a comparison operation between pixels at the same coordinates in a line signal (read data (1) in Figure 1) stored in line memory 14-1, which was read from frame memory 11-1 during a certain period, and a line signal (read data (2) in Figure 1) stored in line memory 14-2, which was read from frame memory 11-2 during the same period. The pixel comparison operation is, for example, the overlay operation described above.

[0026] The output line memory 17 is a third storage unit that stores the processing results (comparison calculation results in Figure 1) of line-unit signals for a corresponding period for a continuous period. For example, the output line memory 17 stores the line-unit processing results of the processing unit 16 for two lines corresponding to a continuous period based on the write enable signal (3) from the control unit 15, and reads out the line-unit processing results (read data in Figure 1) corresponding to the previous period based on the read enable signal (3) from the control unit 15. In the configuration example shown in Figure 1, the signal processing device 1 is equipped with an output line memory 17, but the output line memory may be provided by an external device separate from the signal processing device 1. In this case, the output line memory provided by the external device is connected to the control unit 15, processing unit 16, and switching unit 18 provided by the signal processing device 1.

[0027] The switching unit 18 switches between the output from the processing unit 16 and the output from the output line memory 17 based on a switching signal from the control unit 15. For example, if the control unit 15 completes reading line-by-line signals from all frame memories 11-1 and 11-2 within the period, it controls the switching unit 18 to read the processing results for the previous period stored in the output line memory 17 and switches to the read processing results. Also, if the control unit 15 fails to complete reading signals from at least one of the frame memories 11-1 and 11-2 within the period, it controls the switching unit 18 to switch to the processing results processed by the processing unit 16 after the reading of the unread signals is completed. The processing result switched by the switching unit 18 is output from the signal processing device 1 as a video signal (1).

[0028] The signal processing device 1 is implemented, for example, by a computer. In this case, the computer's memory stores programs that constitute information processing applications for realizing the functions of the synchronization signal generation unit 12, the reading units 13-1 to 13-3, the control unit 15, the processing unit 16, and the switching unit 18. The computer's processor executes the information processing applications read from the memory, thereby realizing the functions of the synchronization signal generation unit 12, the reading units 13-1 to 13-3, the control unit 15, the processing unit 16, and the switching unit 18.

[0029] Next, the signal processing method according to Embodiment 1 will be described. Figure 4 is a flowchart showing the signal processing method according to Embodiment 1, illustrating the flow of operation by the signal processing device 1. The read units 13-1 to 13-3 read video signals line by line from frame memories 11-1 to 11-3 via the AXI bus 3, at intervals corresponding to line units (step ST1). The read unit 13-1 stores the line-by-line signals read from frame memory 11-1 in line memory 14-1 via the AXI bus 3. The read unit 13-2 stores the line-by-line signals read from frame memory 11-2 in line memory 14-2 via the AXI bus 3. At this time, the control unit 15 determines whether the reading of line-by-line signals by the read units 13-1 and 13-2 has been completed within the period corresponding to the line unit (step ST2).

[0030] If the line-by-line signal reading by the reading units 13-1 and 13-2 is completed within the period corresponding to the line (step ST2; YES), the control unit 15 notifies the switching unit 18 of this fact. The processing unit 16 processes the line-by-line signals for the corresponding periods stored in the line memories 14-1 and 14-2 (step ST3). The output line memory 17 stores the processing results of the line-level signals output from the processing unit 16 for the corresponding period, for a continuous period of time. When the control unit 15 notifies the switching unit 18 that the line-by-line signal reading by the reading units 13-1 and 13-2 has been completed within the period corresponding to each line, the switching unit 18 switches to the output of the output line memory 17. As a result, the signal processing device 1 outputs the processing result corresponding to the previous period from among the processing results stored in the output line memory 17 as a video signal (1).

[0031] On the other hand, if the reading of the signal for at least one of the reading units 13-1 and 13-2 is not completed within the period corresponding to the line unit (step ST2; NO), the control unit 15 notifies the switching unit 18 of this fact and controls the reading units 13-1 and 13-2 to stop reading the signal corresponding to the next period (step ST4). At this time, the control unit 15 determines whether the reading of the signals that were not read incomplete has been completed (step ST5). If the reading of the signals that were not read incomplete has not been completed (step ST5; NO), the control unit 15 controls the reading units 13-1 and 13-2 to continue to stop the reading of the signals corresponding to the next period.

[0032] If the control unit 15 has completed reading the signals that were not read in progress (step ST5; YES), it controls the reading units 13-1 and 13-2 to start reading the signals corresponding to the next period (step ST6). When the control unit 15 notifies the switching unit 18 that the reading of the signal for at least one of the line units 13-1 and 13-2 has not been completed within the period corresponding to that line unit, the switching unit 18 switches to the output of the processing unit 16. As a result, the signal processing device 1 outputs the processing result corresponding to the previous period by the processing unit 16 as a video signal (1).

[0033] Figure 5 shows an example of the timing of video display for one frame of a video signal, illustrating the timing for displaying 1920 x 1080 pixel image data in a display with a refresh rate of 60Hz. As shown in Figure 5, the position and size of the active image actually displayed on the display screen, as well as the blanking period during which no image is displayed, are determined based on Vsync, which indicates the timing when one frame transfer is completed, and Hsync, which indicates the timing when one line of data transfer is completed. For example, the effective image size is 1920 pixels horizontally and 1080 pixels vertically. The effective image is displayed in the area enclosed by the blanking period when no image is displayed.

[0034] Figure 6 is a conceptual diagram showing an overview of the process in Embodiment 1 for outputting the results of a comparison operation between pixels at the same coordinates of the video signals read from frame memories 11-1 and 11-2 as a video signal. In Figure 6, the two figures drawn above the arrows are images (1920 x 1080 pixels) of one frame read from frame memories 11-1 and 11-2, respectively, and are displayed on the display at the video display timing shown in Figure 5.

[0035] The image shown below the arrow in Figure 6 is an image generated by the processing unit 16 comparing both images in one frame pixel by pixel. The processing unit 16 sequentially compares pixels at the same coordinates for each line of both images. For example, in one line starting from the top-left pixel of both images, the pixel value "0000" at the top-left coordinate is compared with the pixel value "0000", and the result of the comparison, "0000", is set to the same position in the top-left of the image shown below the arrow in Figure 6. Subsequently, in one line starting from the top-left pixel of both images, the pixel value "0001" next to the top-left coordinate is compared with the pixel value "0001", and the result of the comparison, "0001", is set to the next position in the top-left of the image shown below the arrow in Figure 6. In this way, the processing unit 16 generates an image in which the value of each pixel is the result of the comparison operation by sequentially comparing pixels of the same coordinates in line units of both images for each frame.

[0036] Furthermore, the comparison operation between pixels may use any processing result. The processing unit 16 performs, for example, overlay image processing. For example, overlay image processing can be described in Verilog as follows. In the Verilog comments below, (1) refers to frame memory 11-1, and (2) refers to frame memory 11-2.

[0037] module video_overlay( input wire clk, input wire rst, input wire[23:0] pixel1, / / (1): background video (8-bit R, 8-bit G, 8-bit B) input wire[23:0] pixel2, / / (2): Overlay video (8-bit R, 8-bit G, 8-bit B) output reg [23:0] output_pixel / / Comparison result (8-bit R, 8-bit G, 8-bit B) ); / / Transparency color of the overlay (here, black: R=0, G=0, B=0) wire[23:0] transparent_color=24'h000000; always @(posedge clk or posedge rst) begin if (rst) begin output_pixel <= 24'b0; end else begin / / If pixel2 is not transparent, display pixel2; otherwise, display pixel1. if (pixel2 != transparent_color) begin output_pixel <= pixel2; end else begin output_pixel <= pixel1; end end end endmodule

[0038] (Under normal operation) Figure 7 is a timing chart showing each signal during normal operation of the signal processing device according to Embodiment 1. The video signals input by input units 2-1 to 2-3 are stored in frame memories 11-1 to 11-3 via the AXI bus 3. The control unit 15 performs line counting in accordance with the video display timing based on Vsync and Hsync generated by the synchronization signal generation unit 12. The line count value corresponds to the readout period for each signal on a line basis. In the example in Figure 7, the control unit 15 asserts a transfer start signal (1) to the readout unit 13-1 and a transfer start signal (2) to the readout unit 13-2 at a significant timing of Hsync at the line count value "18".

[0039] Furthermore, the control unit 15 asserts write-enabled signal (1) and write-enabled signal (2) in order to write the video signals read from frame memories 11-1 and 11-2 by the read units 13-1 and 13-2 to line memories 14-1 and 14-2 on a line-by-line basis, and asserts read-enabled signal (1) and read-enabled signal (2) in order to read the signals from line memories 14-1 and 14-2 on a line-by-line basis.

[0040] The read unit 13-1 reads line-by-line write data (1) from the frame memory 11-1 via the AXI bus 3 based on the write enable signal (1), and writes the write data (1) to the line memory 14-1. At this time, the read unit 13-1 outputs a transfer completion signal (1) to the control unit 15 indicating that the transfer of one line has been completed. The line memory 14-1 stores video signals for two lines. Based on the read-enabled signal (1), the read unit 13-1 reads line-level read data (1) corresponding to the previous read period from the line memory 14-1 and outputs it to the processing unit 16.

[0041] Similarly, the read unit 13-2 reads line-by-line write data (2) from the frame memory 11-2 via the AXI bus 3 based on the write enable signal (2), and writes the write data (2) to the line memory 14-2. At this time, the read unit 13-2 outputs a transfer completion signal (2) to the control unit 15 indicating that the transfer of one line has been completed. Line memory 14-2 stores video signals for two lines, similar to line memory 14-1. Based on the read-enabled signal (2), the read unit 13-2 reads line-level read data (2) corresponding to the previous read period from line memory 14-2 and outputs it to the processing unit 16.

[0042] The above operation is repeated until the line count value reaches "1098". Once 1080 lines of active video signals in one frame have been transferred from frame memories 11-1 and 11-2, the control unit 15 resets the line count value based on Vsync and proceeds to the transfer process for the next frame.

[0043] Once the transfer of one line of video signal to line memories 14-1 and 14-2 is complete, the readout units 13-1 and 13-2 read the signals from line memories 14-1 and 14-2 line by line based on the read-enabled signals (1) and (2) from the control unit 15, and output the read signals to the processing unit 16. In Figure 7, there is no delay in reading the write data (1) from frame memory 11-1 and the write data (2) from frame memory 11-2. Therefore, the processing unit 16 can compare the read data (1) and read data (2) corresponding to the same line count value, as indicated by "Processing ○" in Figure 7.

[0044] Figure 8 is a timing chart illustrating the overview of the pixel-level comparison operation. In Figure 8, write data (1) and write data (2) are data for one line consisting of 1920 pixels from the 0th to the 1919th pixel. Read data (1) and read data (2) are the data of the previous line stored in line memory 14-1 and line memory 14-2, and similarly, are data for one line consisting of 1920 pixels from the 0th to the 1919th pixel.

[0045] The processing unit 16 performs a comparison operation between pixels at the same coordinates in read data (1) and read data (2). For example, the 0th pixel in each line is the same pixel, so the 0th pixel value "100000" in read data (1) and the 0th pixel value "200000" in read data (2) are compared. The comparison operation calculates a value indicating whether the two data match or not, and the processing result is output as "R00000". The comparison operation is performed for all pixels in all lines. This generates an image of the processing result, for example, image C in Figure 2.

[0046] Furthermore, data transfer from frame memory 11-3 will also be explained. The readout unit 13-3 reads the video signal from the frame memory 11-3 based on Vsync, Hsync, and Valid generated by the synchronization signal generation unit 12, and performs predetermined processing on the readout video signal. The processed video signal is output from the signal processing device 1 as video signal (2).

[0047] (In case of transfer delay) Figure 9 is a timing chart showing each signal of the signal processing device according to Embodiment 1 when reading is not completed within the reading period for one line. Figure 9 shows the case when the reading of line-by-line signals from frame memory 11-1 is not completed within the period corresponding to one line. The period corresponding to one line is the time defined by the horizontal total size of 2222 clocks, which is the sum of the horizontal effective size (1920) and the horizontal blanking (40 + 162) shown in Figure 5. The reading of the signal corresponding to line count value "19" from frame memory 11-1 has not been completed by the timing corresponding to line count value "20".

[0048] Such delays are due to the fact that AXI bus 3 is a bus interface based on a handshake. For example, in AXI bus 3, if access to frame memory 11-3 has already been performed, it may take time for the transfer response from frame memory 11-1 to occur. At this time, the reading of the signal from frame memory 11-2 has been performed and completed at the intended timing, and the transfer of the next line is ready to begin. However, if the control unit 15 observes the status of the transfer completion signal (1) and the transfer completion signal (2) and finds that neither has been completed, it does not assert either the transfer start signal (1) or the transfer start signal (2), as shown in Figure 9. In other words, when the transfer completion signal (1) and transfer completion signal (2) indicate that the transfer is complete, the control unit 15 asserts the transfer start signal (1) and transfer start signal (2) corresponding to the next line at a significant timing in Hsync.

[0049] Furthermore, the control unit 15 does not assert the read enable signal (1) and read enable signal (2) to line memory 14-1 and line memory 14-2 so that reading is stopped when the line count value is "20". Then, once the reading of the signal ("0001") corresponding to the line count value "19," which had not been read in progress, is completed, the control unit 15 asserts the read enable signal (1) and the read enable signal (2) at the timing of the line count value "21," which is the timing for asserting the transfer start signal (1) and the transfer start signal (2). In other words, the control unit 15 asserts the read-enabled signal (1) and read-enabled signal (2) after both the transfer completion signal (1) and the transfer completion signal (2) have reached the transfer completion state.

[0050] As a result of the above operation, at the timing of line count value "20", the processing unit 16 outputs an unintended processing result. That is, due to the cessation of reading from line memories 14-1 and 14-2, the comparison operation between data corresponding to the timing of line count value "20" is not performed, as shown by "Processing ×" in Figure 9. However, for line count values ​​"21" and beyond, even though the signal corresponding to line count value "19" is delayed until the timing of line count value "20", the processing unit 16 sequentially performs the intended comparison operations, including comparison operations between the data corresponding to line count value "19", and outputs the results of these operations.

[0051] (Switching operation) Figure 10 is a timing chart showing the signals from the output line memory 17 and the switching unit 18 during the normal operation of the signal processing device 1. Figure 11 is a timing chart showing the signals from the output line memory 17 and the switching unit 18 when reading is not completed within the reading period for one line. The output line memory 17 stores the comparison calculation results for two consecutive lines. Based on the write enable signal (3) from the control unit 15, the comparison calculation results from the processing unit 16 are written to the output line memory 17. Furthermore, based on the read enable signal (3) from the control unit 15, the comparison calculation result of the previous line is read from the output line memory 17. As shown in Figures 10 and 11, the output line memory 17 can read the comparison calculation result corresponding to the previous line while writing the comparison calculation result on a line-by-line basis.

[0052] The switching unit 18 switches the output of the output line memory 17 and the output of the processing unit 16 based on the switching signal from the control unit 15. In normal operation, where there is no delay in reading out the signal line by line from frame memories 11-1 and 11-2, as shown in Figure 10, the switching unit 18 switches to the output of the output line memory 17 based on a low-level ("L" level) switching signal, and outputs the read data corresponding to the previous line from the output line memory 17 as video output (1).

[0053] If the reading of the signal corresponding to line count value "21" (data "3" shown in Figure 11) is not completed by the timing corresponding to line count value "22", that is, if a transfer delay of one or more lines occurs, the switching unit 18 switches to the output of the processing unit 16 based on a high-level ("H" level) switching signal from the control unit 15, as shown in Figure 11. As a result, the comparison calculation result corresponding to the timing of the delayed line count value "21" is directly output from the processing unit 16 as video output (1). By controlling the above switching operation, even if a transfer delay of one or more lines occurs, the video signal can be displayed without interruption of a single pixel or line, and at the video display timing shown in Figure 5, as originally intended.

[0054] As described above, the signal processing device 1 according to Embodiment 1 includes: a reading unit 13-1, 13-2 that reads signals stored in frame memories 11-1, 11-2 at periodic intervals corresponding to line units and stores the line-unit signals read from frame memories 11-1 to 11-3 in line memories 14-1, 14-2; a control unit 15 that controls the reading units 13-1 to 13-3 to stop reading line-unit signals corresponding to the next period from frame memories 11-1 to 11-3 if the reading of line-unit signals from at least one of the frame memories 11-1 to 11-3 is not completed within the period corresponding to that line unit, and starts reading line-unit signals corresponding to the next period after the reading of the unread signals is completed; and a processing unit 16 that processes line-unit signals of corresponding periods stored in line memories 14-1, 14-2. As a result, the signal processing device 1 can process the signals to be processed even if there is a delay in reading line-unit signals from either frame memory 11-1 or 11-2.

[0055] The signal processing device 1 according to Embodiment 1 includes a switching unit 18 that switches between the output from the processing unit 16 and the output from the output line memory 17, which stores the processing results of line-unit signals for a corresponding period for a continuous period. If the reading of line-unit signals from at least one frame memory is not completed within the period, the processing unit 16 processes the line-unit signals for the corresponding period after the reading of the unread signals is completed. If the reading of line-unit signals from all frame memories 11-1 and 11-3 is completed within the period, the control unit 15 controls the switching unit 18 to read from the processing results for the previous period stored in the output line memory 17 and switch to the read processing results. If the reading of signals from at least one frame memory is not completed within the period, the switching unit 18 controls the switching unit 18 to switch to the processing results processed by the processing unit 16 after the reading of the unread signals is completed. As a result, the signal processing device 1 can output the processing results of the signals to be processed continuously even if the reading of line-unit signals from any of the frame memories 11-1 and 11-2 is delayed.

[0056] In the signal processing device 1 according to Embodiment 1, the processing unit 16 performs a comparison operation between pixels at the same coordinates in a line-by-line video signal. As a result, the signal processing device 1 can generate an image by overlaying multiple video signals.

[0057] In the signal processing method according to Embodiment 1, if the control unit 15 fails to complete the reading of a line-unit signal from at least one of the frame memories 11-1 to 11-3 within the period corresponding to that line unit, it controls the reading units 13-1 to 13-3 to stop reading the line-unit signal corresponding to the next period from the frame memories 11-1 to 11-3, and starts reading the line-unit signal corresponding to the next period after the reading of the unread signal has been completed. By executing the above method, the signal processing device 1 can process the signals to be processed even if there is a delay in reading a line-unit signal from either frame memory 11-1 or 11-2.

[0058] In the signal processing method according to Embodiment 1, if the control unit 15 completes the reading of line-by-line signals from all frame memories 11-1 to 11-3 within the specified period, it controls the switching unit 18 to read the processing result from the previous period stored in the output line memory 17 and switch to the read processing result. If the reading of signals from at least one frame memory is not completed within the specified period, the control unit 18 controls the switching unit 18 to switch to the processing result processed by the processing unit 16 after the reading of the unread signals is completed. By performing the above method, the signal processing device 1 can output the processing results of the signals to be processed consecutively, even if there is a delay in reading line-by-line signals from either frame memory 11-1 or 11-2.

[0059] Embodiment 2. Embodiment 2 describes a signal processing device that includes an output frame memory instead of the output line memory 17 and the switching unit 18. Figure 12 is a block diagram showing an example configuration of the signal processing device 1A according to Embodiment 2, where components that function similarly to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. As shown in Figure 12, the signal processing device 1A is a device connected to the input units 2-1 to 2-3 via the AXI bus 3, and includes frame memories 11-1 to 11-3, a synchronization signal generation unit 12, readout units 13-1 to 13-3, line memories 14-1 and 14-2, a control unit 15A, a processing unit 16, and an output frame memory 19.

[0060] The output frame memory 19 is a fourth storage unit that stores a frame of video signals, which are first signals composed of multiple line-unit signals. For example, the output frame memory 19 stores the processing results for each line output from the processing unit 16 in response to a write control signal from the control unit 15A, and the video signal for one frame, composed of the processing results for each line, is read out line by line as video signal (1) in response to a read control signal from the control unit 15A. In the configuration example shown in Figure 12, the signal processing unit 1A is equipped with an output frame memory 19. However, the output frame memory may be provided by an external device separate from the signal processing unit 1A. In this case, the output frame memory provided by the external device is connected to the control unit 15 and processing unit 16 of the signal processing unit 1A.

[0061] The control unit 15A stores the processing results output from the processing unit 16 in the output frame memory 19 and controls the reading of the video signal from the output frame memory 19 line by line. Furthermore, the control unit 15A, similar to the control unit 15, controls the reading of signals from frame memories 11-1 and 11-2 by the reading units 13-1 and 13-2, and the writing and reading of signals to line memories 14-1 and 14-2, based on Vsync, Hsync, and Valid generated by the synchronization signal generation unit 12.

[0062] The signal processing device 1A is implemented, for example, by a computer. In this case, the computer's memory stores programs that constitute information processing applications for realizing the functions of the synchronization signal generation unit 12, the reading units 13-1 to 13-3, the control unit 15A, and the processing unit 16. The computer's processor executes the information processing applications read from the memory, thereby realizing the functions of the synchronization signal generation unit 12, the reading units 13-1 to 13-3, the control unit 15A, and the processing unit 16.

[0063] (Under normal operation) Figure 13 is a timing chart showing the signals from the output frame memory 19 during the normal operation of the signal processing device 1A. The video signals input by input units 2-1 to 2-3 are stored in frame memories 11-1 to 11-3 via the AXI bus 3. The control unit 15A performs line counting in accordance with the video display timing based on Vsync and Hsync generated by the synchronization signal generation unit 12. The line count value corresponds to the readout period for each signal on a line basis. In the example in Figure 13, the control unit 15A asserts a transfer start signal (1) to the readout unit 13-1 and a transfer start signal (2) to the readout unit 13-2 at a significant timing of Hsync when the line count value is "18".

[0064] Furthermore, the control unit 15A asserts write-enabled signal (1) and write-enabled signal (2) in order to write the video signals read from frame memories 11-1 and 11-2 by the read units 13-1 and 13-2 to line memories 14-1 and 14-2 on a line-by-line basis, and asserts read-enabled signal (1) and read-enabled signal (2) in order to read the signals from line memories 14-1 and 14-2 on a line-by-line basis.

[0065] The read unit 13-1 reads line-by-line write data (1) from the frame memory 11-1 via the AXI bus 3 based on the write enable signal (1), and writes the write data (1) to the line memory 14-1. At this time, the read unit 13-1 outputs a transfer completion signal (1) to the control unit 15A indicating that the transfer of one line has been completed. The line memory 14-1 stores video signals for two lines. Based on the read-enabled signal (1), the read unit 13-1 reads line-level read data (1) corresponding to the previous read period from the line memory 14-1 and outputs it to the processing unit 16.

[0066] Similarly, the read unit 13-2 reads line-by-line write data (2) from the frame memory 11-2 via the AXI bus 3 based on the write enable signal (2), and writes the write data (2) to the line memory 14-2. At this time, the read unit 13-2 outputs a transfer completion signal (2) to the control unit 15A indicating that the transfer of one line has been completed. Line memory 14-2 stores video signals for two lines, similar to line memory 14-1. Based on the read-enabled signal (2), the read unit 13-2 reads line-level read data (2) corresponding to the previous read period from line memory 14-2 and outputs it to the processing unit 16.

[0067] The above operation is repeated until the line count value reaches "1098". Once 1080 lines of active video signals in one frame have been transferred from frame memories 11-1 and 11-2, the control unit 15A resets the line count value based on Vsync and proceeds to the transfer process for the next frame.

[0068] Once the transfer of one line of video signal to line memories 14-1 and 14-2 is complete, the readout units 13-1 and 13-2 read the signals from line memories 14-1 and 14-2 line by line based on the read-enabled signals (1) and (2) from the control unit 15A, and output the read signals to the processing unit 16. In Figure 13, there is no delay in reading the write data (1) from frame memory 11-1 and the write data (2) from frame memory 11-2. Therefore, the processing unit 16 can compare the read data (1) and read data (2) that correspond to the same line count value.

[0069] Furthermore, the control unit 15A generates a line write address and a write enable signal for the output frame memory 19, and controls the system so that the line-by-line comparison calculation results output from the processing unit 16 are written to the output frame memory 19. Furthermore, the control unit 15A generates line read addresses for the output frame memory 19 and controls it so that the video signal from the previous frame stored in the output frame memory 19 is read out line by line. The signals read out line by line from the output frame memory 19 are output as video signals (1) from the signal processing device 1A.

[0070] (In the case of a 1-line transfer delay) Figure 14 is a timing chart showing each signal from the output frame memory 19 when reading is not completed within the read period for one line. Figure 14 shows the case where reading a line-by-line signal from frame memory 11-2 is not completed within the period corresponding to one line. The period corresponding to one line is the time defined by the horizontal total size of 2222 clocks, which is the sum of the horizontal effective size (1920) and the horizontal blanking (40 + 162) shown in Figure 5. The reading of the signal from frame memory 11-2 corresponding to line count value "21" is not completed until the timing corresponding to line count value "22".

[0071] Such delays are due to the fact that the AXI bus 3 is a bus interface based on a handshake, as shown in Embodiment 1. For example, in the AXI bus 3, if access to the frame memory 11-3 has already been performed, it may take some time for the transfer response from the frame memory 11-2 to occur. At this time, the reading of the signal from the frame memory 11-1 has been performed and completed at the intended timing, and the transfer of the next line can begin. However, if the control unit 15A observes the status of the transfer completion signal (1) and the transfer completion signal (2) and finds that neither has been completed, it does not assert either the transfer start signal (1) or the transfer start signal (2). In other words, when the transfer completion signal (1) and transfer completion signal (2) indicate that the transfer is complete, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) corresponding to the next line at a suitable timing in Hsync.

[0072] Furthermore, the control unit 15A does not assert the read enable signal (1) and read enable signal (2) to line memory 14-1 and line memory 14-2 so that reading is stopped at the timing of the line count value "22". Then, once the reading of the signal corresponding to the line count value "21", which had not been read in progress, is completed, the control unit 15A asserts the read enable signal (1) and the read enable signal (2) at the timing of the line count value "23", which asserts the transfer start signal (1) and the transfer start signal (2). In other words, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) after both the transfer completion signal (1) and the transfer completion signal (2) have reached the transfer completion state, and then asserts the read enable signal (1) and the read enable signal (2).

[0073] Due to the above operation, at the timing of the line count value "22", the processing unit 16 outputs an unintended processing result. Therefore, the control unit 15A stops and holds the increment of the line write address and stops writing the comparison calculation result from the processing unit 16 to the output frame memory 19 (stops writing). However, for line count values ​​of "23" and above, the control unit 15A resumes incrementing the line write address. As a result, even though there was a delay in reading the signal corresponding to line count value "21" (the data "1003" in Figure 14), the processing unit 16 sequentially performs the intended comparison operations, including comparison operations between the data corresponding to line count value "21," and the results of these operations are written to the output frame memory 19 line by line.

[0074] (In the case of a 2-line transfer delay) Figure 15 is a timing chart showing each signal from the output frame memory 19 when the reading of two lines is not completed. In the example shown in Figure 15, the reading of the signal corresponding to line count value "21" (data "1003" in Figure 15) from frame memory 11-2 is not completed by the timing corresponding to line count value "22", and the reading of the signal corresponding to line count value "23" (data "1004" in Figure 15) from frame memory 11-1 is not completed by the timing corresponding to line count value "24".

[0075] As shown in Figure 15, at the timing corresponding to the line count value "21", the signal read from frame memory 11-1 has been completed at the intended timing, and the transfer of the next line is ready to begin. However, if the control unit 15A observes the status of the transfer completion signal (1) and the transfer completion signal (2) and finds that neither has been completed, it does not assert either the transfer start signal (1) or the transfer start signal (2). In other words, when the transfer completion signal (1) and transfer completion signal (2) indicate that the transfer is complete, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) corresponding to the next line at a suitable timing in Hsync.

[0076] Furthermore, the control unit 15A does not assert the read enable signal (1) and read enable signal (2) to line memory 14-1 and line memory 14-2 so that reading is stopped at the timing of the line count value "22". Then, once the reading of the signal corresponding to the line count value "21", which had not been read in progress, is completed, the control unit 15A asserts the read-enabled signal (1) and the read-enabled signal (2) at the timing of the line count value "21" which asserts the transfer start signal (1) and the transfer start signal (2). In other words, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) after both the transfer completion signal (1) and the transfer completion signal (2) have reached the transfer completion state, and then asserts the read enable signal (1) and the read enable signal (2).

[0077] Due to the above operation, at the timing of the line count value "22", the processing unit 16 outputs an unintended processing result. Therefore, the control unit 15A stops and holds the increment of the line write address and stops writing the comparison calculation result from the processing unit 16 to the output frame memory 19 (stops writing). However, at the timing corresponding to line count value "23", the control unit 15A resumes incrementing the line write address. As a result, even though there was a delay in reading the signal corresponding to line count value "21" (the data "1003" in Figure 15), the processing unit 16 sequentially performs the intended comparison operations, including comparison operations between the data corresponding to line count value "21", and the results of these operations are written to the output frame memory 19 line by line.

[0078] Furthermore, as shown in Figure 15, at the timing corresponding to the line count value "23", the signal read from frame memory 11-2 has been completed at the intended timing, and the transfer of the next line is ready to begin. However, if the control unit 15A observes the status of the transfer completion signal (1) and the transfer completion signal (2) and finds that neither has been completed, it does not assert either the transfer start signal (1) or the transfer start signal (2). In other words, when the transfer completion signal (1) and transfer completion signal (2) indicate that the transfer is complete, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) corresponding to the next line at a suitable timing in Hsync.

[0079] Furthermore, the control unit 15A does not assert the read enable signal (1) and read enable signal (2) to line memory 14-1 and line memory 14-2 so that reading is stopped at the timing of the line count value "24". Then, once the reading of the signal corresponding to the line count value "23," which had not been read in progress, is completed, the control unit 15A asserts the read-enabled signal (1) and the read-enabled signal (2) at the timing of the line count value "25," which is when the transfer start signal (1) and the transfer start signal (2) are asserted. In other words, the control unit 15A asserts the transfer start signal (1) and transfer start signal (2) after both the transfer completion signal (1) and the transfer completion signal (2) have reached the transfer completion state, and then asserts the read enable signal (1) and the read enable signal (2).

[0080] Due to the above operation, at the timing of the line count value "24", the processing unit 16 outputs an unintended processing result. Therefore, the control unit 15A stops and holds the increment of the line write address and stops writing the comparison calculation result from the processing unit 16 to the output frame memory 19 (stops writing). However, at the timing corresponding to the line count value "25", the control unit 15A resumes incrementing the line write address. As a result, even though there was a delay in reading the signal corresponding to the line count value "23" (the data "1004" in Figure 15), the processing unit 16 sequentially performs the intended comparison operations, including the comparison operation between the data corresponding to the line count value "23", and the results of these operations are written to the output frame memory 19 line by line.

[0081] By controlling the signal processing unit 1A to stop the write operation at the timing of line count values ​​"22" and "24" where delays occur, it is possible to output video signals continuously even when delays occur on two lines. Furthermore, within the vertical blanking period (for example, the sum of the upper vertical blanking period of 20 lines and the lower vertical blanking period of 25 lines shown in Figure 5), even if transfer delays occur on multiple lines, it is possible to output video signals without interruption, not even for a single pixel or line, just as in normal operation.

[0082] As described above, in the signal processing device 1A according to Embodiment 2, the control unit 15A stores the video signal, which is composed of multiple line-unit signals read out at intervals, in the output frame memory 19, and reads out the video signal line by line from the output frame memory 19. As a result, the signal processing device 1A can process the signals to be processed even if there is a delay in reading line-by-line signals from either frame memory 11-1 or 11-2. Furthermore, even if a transfer delay occurs, the signal processing device 1A can output the video signal without interruption, not even for a single pixel or line, just as in normal operation.

[0083] In the signal processing device 1A according to Embodiment 2, the processing unit 16 performs a comparison operation between pixels at the same coordinates in the line-by-line video signals. As a result, the signal processing device 1A can generate an image by overlaying multiple video signals.

[0084] In the signal processing method according to Embodiment 2, the control unit 15A stores the video signal, which is composed of signals read out at each time period, in the output frame memory 19, and reads out the video signal from the output frame memory 19 line by line. By having the signal processing device 1A perform the above method, even if a transfer delay occurs, it is possible to output the video signal without interruption, not even a single pixel or line, just as in normal operation.

[0085] In Embodiments 1 and 2, the transfer start signal was described separately as transfer start signal (1) and transfer start signal (2) for convenience of explanation, but it is not limited to this. For example, if it is asserted based on the observation result of the transfer completion state, the transfer start signal may be a single control signal.

[0086] Embodiments 1 and 2 show a display with a refresh rate of 60Hz that displays image data with 1920 x 1080 pixels per frame, but are not limited to this. For example, the signal processing unit 1 and signal processing unit 1A are also applicable when displaying image data with 1280 x 960 pixels per frame on a display with a refresh rate of 85 Hz, or when displaying image data with 1600 x 1200 pixels per frame on a display with a refresh rate of 75 Hz. In signal processing devices 1 and 1A, the same effect can be obtained with any other video display timing, as long as it has a vertical blanking period.

[0087] Embodiments 1 and 2 demonstrate the transfer of video signals via the AXI bus, but are not limited to this. For example, equivalent effects can be obtained with other bus interfaces that have handshake functionality, not just the AXI bus.

[0088] In Embodiments 1 and 2, the first storage unit for storing signals processed by the processing unit 16 is shown to consist of two frame memories, 11-1 and 11-2, but the invention is not limited to this. For example, the same effect can be obtained even if there are three or more first storage units.

[0089] In Embodiments 1 and 2, the processing unit 16 was shown to process pixels at the same coordinates on a single line. However, the processing targets do not have to be pixels at the same coordinates, as long as the signals are on a line-by-line basis. Figure 16 is a conceptual diagram showing an overview of a modified version of the process in Embodiment 1 and Embodiment 2, which outputs the result of a comparison operation between pixels at the same coordinates of the video signals read from frame memories 11-1 and 11-2 as a video signal. The two figures drawn in Figure 16 are images (1920 x 1080 pixels) of one frame read from frame memories 11-1 and 11-2, respectively, and are displayed on the display at the video display timing shown in Figure 5.

[0090] For example, the processing unit 16 may compare one pixel in the video signal read from frame memory 11-1 with two pixels adjacent to the coordinates of the aforementioned pixel in the video signal read from frame memory 11-2. In Figure 16, the pixel value "0000" at the top-left coordinate of the line starting from the top-left pixel of both images is compared with the pixel value "0000" at the same coordinate. On the other hand, the pixel value "0001" at the adjacent coordinate is compared with the pixel values ​​"0000" and "0002" on either side of the same coordinate. Furthermore, the pixel value "0002" of the adjacent coordinate is compared with the pixel values ​​"0001" and "0003" of the pixels on either side of the same coordinate. The results of these comparison operations generate the video signal (1). This allows comparison calculations with the intended target to be performed even if there is a delay in reading signals from either frame memory 11-1 or 11-2.

[0091] In Embodiments 1 and 2, the signal being handled was a video signal. However, the signal processing devices 1 and 1A can be applied to any system that reads data from multiple first storage units over a certain period of time and processes it at periodic intervals corresponding to a certain processing unit. The same effect can be obtained with other data as well as video signals.

[0092] Furthermore, it is possible to combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. [Explanation of Symbols]

[0093] 1,1A Signal processing unit, 2-1~2-3 Input unit, 3 AXI bus, 11-1~11-3 Frame memory, 12 Synchronization signal generation unit, 13-1~13-3 Read unit, 14-1,14-2 Line memory, 15,15A Control unit, 16 Processing unit, 17 Output line memory, 18 Switching unit, 19 Output frame memory.

Claims

1. A reading unit reads signals stored in a plurality of first storage units at periodic intervals corresponding to a certain processing unit, and stores the signals of the processing unit read from the plurality of first storage units in a plurality of second storage units. A control unit controls the read unit to stop reading the signal for the next period if the reading of the signal for the processing unit from at least one of the multiple first storage units is not completed within the period corresponding to the processing unit, and to start reading the signal for the next period after the reading of the uncompleted signal is completed. The system comprises a processing unit that processes signals of the processing units for corresponding periods stored in a plurality of the second storage units. A signal processing device characterized by the following:

2. The system includes a switching unit that switches between the output from the processing unit and the output from a third storage unit that stores the processing results of the signals of the processing units for the corresponding period for a continuous period. If the reading of the signal of the processing unit from at least one of the first storage units is not completed within the period corresponding to the processing unit, the processing unit shall process the signals of the processing units for the corresponding period after the reading of the unread signals has been completed. The control unit, When the reading of signals for the processing units from all of the first storage units is completed within the period corresponding to the processing unit, the switching unit is controlled to switch to the processing result for the period read from the third storage unit. If the reading of signals from at least one of the first storage units is not completed within the specified period, the switching unit is controlled to switch to the processing result processed by the processing unit after the reading of the unread signals has been completed. The signal processing device according to claim 1.

3. The control unit stores a first signal, which consists of signals from a plurality of processing units read out for each period, in the fourth storage unit, and reads out the first signal from the fourth storage unit for each processing unit. The signal processing device according to claim 1.

4. The first memory unit described above is a frame memory that stores video signals on a frame-by-frame basis. The reading unit reads the video signals stored in the plurality of frame memories at periodic intervals corresponding to the line units which are the processing units. The second storage unit is a line memory that stores the line-unit signals read from each of the multiple frame memories by the reading unit, The control unit, If the reading of a signal from at least one of the multiple frame memories is not completed within the period corresponding to the line unit, the reading unit is controlled to stop the reading of the line unit signal corresponding to the next period from the multiple frame memories, and after the reading of the unread signals is completed, the reading of the line unit signal corresponding to the next period is started. The processing unit processes the line-unit signals for corresponding periods stored in the multiple line memories. The signal processing device according to claim 1.

5. The first memory unit described above is a frame memory that stores video signals on a frame-by-frame basis. The reading unit reads the video signals stored in the plurality of frame memories at periodic intervals corresponding to the line units which are the processing units. The second storage unit is a line memory that stores the line-unit signals read from each of the multiple frame memories by the reading unit, The third storage unit is an output line memory that stores the processing results of the line-unit signals for the corresponding period processed by the processing unit for a continuous period of time. The switching unit switches between the output from the processing unit and the output from the output line memory. If the reading of at least one signal from the frame memory is not completed within the period corresponding to that line unit, the processing unit shall process the signals of the line units for the corresponding period after the reading of the unread signals has been completed. The control unit, When the reading of the line-by-line signals from all the frame memories is completed within the period, the switching unit is controlled to read the processing results for the previous period stored in the output line memory and switch to the read processing results. If the reading of signals from at least one of the frame memories is not completed within the specified period, the switching unit is controlled to switch to the processing result processed by the processing unit after the reading of the unread signals has been completed. The signal processing apparatus according to claim 2.

6. The first memory unit described above is a frame memory that stores video signals on a frame-by-frame basis. The reading unit reads the video signals stored in the plurality of frame memories at periodic intervals corresponding to the line units which are the processing units. The second storage unit is a line memory that stores the line-unit signals read from each of the multiple frame memories by the reading unit, The fourth storage unit is an output frame memory that stores the video signal, which is composed of a plurality of frame-unit signals read out at each time period, as the first signal. The control unit stores the video signal, which is composed of signals read out for each period, in the output frame memory, and reads out the video signal from the output frame memory in line units. The signal processing apparatus according to claim 3.

7. The processing unit performs a comparison operation on pixels at the same coordinates in the video signal on a line-by-line basis. The signal processing apparatus according to any one of claims 4 to 6.

8. A signal processing method using the signal processing device described in claim 1, The control unit, If the reading of the signal for the processing unit from at least one of the multiple first storage units is not completed within the period corresponding to the processing unit, the reading unit is controlled to stop reading the signal for the processing unit corresponding to the next period from the multiple first storage units. After the reading of any signals that were not yet completed is finished, the reading of the signals for the processing unit corresponding to the next period is started. A signal processing method characterized by the following:

9. A signal processing method using the signal processing device described in claim 2, The control unit, When the reading of signals for the processing units from all of the first storage units is completed within the period corresponding to the processing unit, the switching unit is controlled to read the processing results for the next period stored in the third storage unit and switch to the read processing results. If the reading of signals from at least one of the first storage units is not completed within the specified period, the switching unit is controlled to switch to the processing result processed by the processing unit after the reading of the unread signals has been completed. A signal processing method characterized by the following:

10. A signal processing method using the signal processing device described in claim 3, The control unit stores the first signal, which is composed of signals read out at each time period, in the fourth storage unit, and reads out the first signal from the fourth storage unit in the processing unit. A signal processing method characterized by the following:

Images