Remote control device

Abstract

To solve such a problem that it is difficult to easily realize remote control.SOLUTION: A remote control device 700 comprises: a reception unit 721 which receives image data at least with a screen display part in an operation object device as a subject; a conversion unit 722 which performs prescribed conversion processing on the basis of information indicating the size of the screen display part in the operation object device and the image data; a character recognition unit 723 which performs character recognition processing to the converted image data generated as a result of the conversion processing by the conversion unit 722; and a display unit 724 which displays the processed image data after the processing by the character recognition unit 723.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present invention relates to a remote operation device, a remote operation method, a program, and a remote operation system.

Background Art

[0002] Techniques used for remotely operating an information processing device such as a personal computer are known.

[0003] For example, Patent Document 1 describes a transmission-side network I / O box device characterized by including a first interface unit that controls an interface with an input device for a local PC, a first change recognition unit, a first compression unit, a first coupling unit, a first network I / O unit, a first splitting unit, a first decompression unit, and a first interface unit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] There are cases where it is desired to remotely operate an information processing device such as a personal computer in a local network environment or an offline environment. In such cases, with the technique described in Patent Document 1, network I / O box devices are required on both the operated side and the operating side, making it difficult to easily perform remote operation. Thus, there has been a problem that it is difficult to easily perform remote operation.

[0006] Therefore, an object of the present invention is to provide a remote operation device, a remote operation method, a program, and a remote operation system for solving the above-described problems.

Means for Solving the Problems

[0007] To achieve this objective, a remote control device, which is one form of the present disclosure, A receiving unit that receives image data in which at least the screen display unit of the operated device is the subject, A conversion unit that performs a predetermined conversion process based on information indicating the size of the screen display unit of the operated device and the image data, A character recognition unit performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit, A display unit that displays the processed image data after processing by the character recognition unit, has This is the structure it takes.

[0008] Furthermore, another form of this disclosure, the remote control method, is Information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. This is the structure it takes.

[0009] Furthermore, other forms of this disclosure, such as program, In an information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. This is a program for performing the processing.

[0010] In addition, a remote operation system, which is another aspect of the present disclosure, a receiving unit that receives image data having at least the screen display unit of the operated device as a subject, an information indicating the size of the screen display unit of the operated device, and a conversion unit that performs a predetermined conversion process based on the information and the image data, a character recognition unit that performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit, and a display unit that displays the processed image data after the processing by the character recognition unit, a remote operation device having the above, an imaging device that acquires the image data having the screen display unit of the operated device as a subject and transmits the acquired image data to the remote operation device, having such a configuration.

Effect of the Invention

[0011] According to each of the above-described configurations, it is possible to easily remotely operate the information processing device.

Brief Description of the Drawings

[0012] [Figure 1] It is a diagram showing a configuration example of a remote operation system in a first embodiment of the present disclosure. [Figure 2] It is a diagram showing another configuration example of the remote operation system. [Figure 3] It is a block diagram showing a configuration example of the remote operation device. [Figure 4] It is a diagram showing an example of the specification information. [Figure 5] It is a diagram showing a processing example of the remote operation device. [Figure 6] It is a diagram showing a processing example of the remote operation device. [Figure 7] It is a flowchart showing an operation example of the remote operation device. [Figure 8] It is a flowchart showing an operation example of the remote operation device. [Figure 9] It is a block diagram showing another configuration example of the remote operation device. [Figure 10]It is a diagram showing an example of the hardware configuration of the remote control device in the second embodiment of the present disclosure. [Figure 11] It is a block diagram showing an example of the configuration of the remote control device.

Embodiments for Carrying Out the Invention

[0013] [First Embodiment] The first embodiment of the present disclosure will be described with reference to FIGS. 1 to 9. FIGS. 1 and 2 are diagrams for explaining the outline of the configuration of the remote control system 100. FIG. 3 is a block diagram showing an example of the configuration of the remote control device 500. FIG. 4 is a diagram showing an example of the specification information 541. FIGS. 5 and 6 are diagrams showing examples of the processing of the remote control device 500. FIGS. 7 and 8 are flowcharts showing examples of the operations of the remote control device 500. FIG. 9 is a block diagram showing another example of the configuration of the remote control device 500.

[0014] In the first embodiment of the present disclosure, a remote control system 100 for remotely controlling an operated device 200 placed in a local network environment or an offline environment will be described. As will be described later, when the remote control system 100 acquires image data of the operated device 200 as a subject using the imaging device 300, it extracts the screen display portion of the operated device 200 shown in the acquired image data, and converts the shape of the extracted screen display portion of the operated device 200 into a predetermined shape. Then, the remote control system 100 executes a character recognition process such as an OCR (Optical Character Recognition) process on the converted image data. After that, the remote control system 100 displays the processed image data, which is the image data after the above processing, on the screen.

[0015] Furthermore, the remote control system 100 displays the processed image data described above on the screen and transmits operation information to the controlled device 200 in response to operations on the input unit such as a keyboard or mouse. For example, the remote control system 100 transmits operation information to the controlled device 200 via a mouse / keyboard receiver 400 for remote control emulation. This allows the controlled device 200 to recognize operations on the mouse, keyboard, etc. The transmission of operation information may be implemented using known technologies.

[0016] Figures 1 and 2 show an example configuration of the remote control system 100. Referring to Figure 1, the remote control system 100 includes, for example, a device to be controlled 200, an imaging device 300, a mouse / keyboard receiver 400 for remote control emulation, and a remote control device 500. As shown in Figure 2, the remote control system 100 may also include a mobile terminal 600 such as a smartphone. As shown in Figures 1 and 2, the remote control device 500 is connected to the imaging device 300 and the mouse / keyboard receiver 400 via a network or the like so that they can communicate with each other.

[0017] The operated device 200 is an information processing device located in a local network environment or an offline environment. For example, the operated device 200 has a screen display unit such as an LCD (Liquid Crystal Display) that displays information. The operated device 200 also recognizes operations on a keyboard, mouse, etc., according to the acquired operation information. For example, the operated device 200 may be a general information processing device such as a personal computer, tablet, or other portable terminal.

[0018] The imaging device 300 is an imaging device such as a camera that acquires image data with at least the screen display unit of the operated device 200 as the subject. For example, the imaging device 300 acquires time-series image data. The imaging device 300 also transmits the acquired image data to the remote control device 500 via a network N or the like.

[0019] For example, the imaging device 300 has a wireless communication function, and can transmit image data to the remote control device 500 using this wireless communication function. The imaging device 300 may also transmit image data to the remote control device 500 using a tethering function or the like of a mobile terminal 600 that is connected to the imaging device 300 in a communicative manner.

[0020] The mouse / keyboard receiver 400 is a remote control emulation receiver that receives operation information such as keyboard and mouse input from the remote control device 500 via the network N. For example, the mouse / keyboard receiver 400 is connected to the controlled device 200 and outputs the operation information received from the remote control device 500 to the controlled device 200. By outputting operation information to the controlled device 200, the controlled device 200 recognizes the operation of the keyboard, mouse, etc.

[0021] For example, the mouse / keyboard receiver 400 has a wireless communication function, and can receive operation information from the remote control device 500 using this wireless communication function. The mouse / keyboard receiver 400 may also receive operation information from the remote control device 500 using a tethering function or the like of a mobile terminal 600 that is communicatively connected to the mouse / keyboard receiver 400.

[0022] The mouse / keyboard receiver 400 may be a known device. For example, the mouse / keyboard receiver 400 may utilize KVM (Keyboard, Video, Mouse) over IP (Internet Protocol).

[0023] The remote control device 500 is an information processing device that remotely controls the device to be controlled 200. Figure 3 shows an example of the configuration of the remote control device 500. Referring to Figure 3, the remote control device 500 has as its main components, for example, an operation input unit 510, a screen display unit 520, a communication I / F unit 530, a storage unit 540, and an arithmetic processing unit 550.

[0024] The operation input unit 510 consists of an operation input device such as a keyboard or mouse. The operation input unit 510 detects the operation of the person operating the remote control device 500 and outputs it to the calculation processing unit 550.

[0025] The screen display unit 520 consists of a screen display device such as an LCD. The screen display unit 520 can display various information stored in the storage unit 540 on the screen in response to instructions from the arithmetic processing unit 550.

[0026] The communication interface unit 530 consists of a data communication circuit. The communication interface unit 530 performs data communication with external devices such as the imaging device 300 and the mouse / keyboard receiver 400, which are connected via a communication line.

[0027] The storage unit 540 is a storage device such as a hard disk or memory. The storage unit 540 stores processing information and programs 544 necessary for various processes in the arithmetic processing unit 550. The programs 544 are read into the arithmetic processing unit 550 and executed to realize various processing functions. The programs 544 are pre-read from external devices or recording media via data input / output functions such as the communication I / F unit 530 and stored in the storage unit 540. The main information stored in the storage unit 540 includes, for example, standard information 541, image data information 542, and processed image data information 543.

[0028] The standard information 541 includes information indicating the size of the screen display unit of the operated device 200. For example, the standard information 541 is acquired in advance from an external device via the communication I / F unit 530 or the like, or is input in advance using the operation input unit 510 or the like, and stored in the storage unit 540.

[0029] Figure 4 shows an example of standard information 541. Referring to Figure 4, standard information 541 shows the height and width of the screen display unit for each size. Thus, standard information 541 indicates the size of the screen display unit of the operated device 200.

[0030] Image data information 542 includes image data received from the imaging device 300. For example, image data information 542 includes time-series image data. Image data information 542 may associate image data with image data identification information, information indicating the date and time the image data was acquired, information indicating the location where the image data was acquired, and other arbitrary information. For example, image data information 542 is updated each time the receiving unit 551 receives image data from an external device such as a camera or an external information processing device.

[0031] The processed image data information 543 includes the processed image data after processing by the conversion unit 553 and the character recognition unit 554, which will be described later. For example, the processed image data information 543 is updated when the character recognition unit 554 performs character recognition processing such as OCR.

[0032] The arithmetic processing unit 550 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 550 reads and executes a program 544 from the storage unit 540, thereby realizing various processing functions by having the hardware and the program 544 work together. The main processing functions realized by the arithmetic processing unit 550 include, for example, a receiving unit 551, an extraction unit 552, a conversion unit 553, a character recognition unit 554, a display unit 555, and an operation information transmission unit 556.

[0033] The receiving unit 551 receives image data and information indicating the size of the image display unit of the operated device 200 from an external device.

[0034] For example, the receiving unit 551 receives image data from the imaging device 300 via the communication I / F unit 530. The receiving unit 551 may also receive information from the imaging device 300 indicating the date and time and location where the image data was acquired, along with the image data. The receiving unit 551 also stores the received image data and other information as image data information 542 in the storage unit 540.

[0035] Furthermore, for example, the receiving unit 551 can obtain information indicating the size of the screen display unit of the operated device 200 from the mobile terminal 600 or other external devices via the communication I / F unit 530. The information indicating the size of the screen display unit may indicate the size of the screen display unit, such as xx type, or it may indicate the height and width of the screen display unit.

[0036] The extraction unit 552 extracts the screen display unit of the operated device 200, which is the subject, from the image data contained in the image data information 542. The extraction unit 552 may extract the screen display unit from the image data using known means.

[0037] For example, the extraction unit 552 extracts the screen display unit using an object detection algorithm such as YOLO (You Only Look Once). The extraction unit 552 may also extract the screen display unit of the operated device 200 using other known methods, such as extracting the screen display unit using an object detector created using OpenCV (Open Source Computer Vision Library), or detecting the screen display unit using a template matching method.

[0038] The conversion unit 553 performs a conversion process to convert the shape of at least the portion of the image data corresponding to the extracted screen display portion, based on the extraction results from the extraction unit 552 and information indicating the size of the screen display portion.

[0039] For example, the conversion unit 553 calculates conversion parameters necessary when converting the shape of the screen display area extracted by the extraction unit 552 into a predetermined shape. For example, the conversion unit 553 calculates conversion parameters necessary when converting the shape of the screen display area in the image data into a shape corresponding to the size of the screen display area. Specifically, for example, the conversion unit 553 calculates a homography transformation matrix as a conversion parameter. The conversion unit 553 may calculate the homography transformation matrix using known means. For example, the conversion unit 553 can calculate the homography transformation matrix by solving a system of equations using the coordinates of the extracted screen display area and the coordinates corresponding to the shape of the screen display area, which are specified according to the size of the screen display area.

[0040] Furthermore, the conversion unit 553 converts the image data from which the screen display area has been extracted by performing calculations using the calculated conversion parameters. For example, the conversion unit 553 generates converted image data in which at least the shape within the region is converted by performing matrix calculations on at least the region corresponding to the screen display area within the image data using the calculated conversion parameters. By performing the conversion, the conversion unit 553 can correct the shape of the screen display area and the display within the image display area in the image data, even in cases where, for example, the imaging position of the imaging device 300 is shifted and the screen display area of ​​the operated device 200 cannot be imaged from directly in front.

[0041] For example, as illustrated above, the conversion unit 553 generates converted image data by performing homography conversion processing based on the extraction results by the extraction unit 552 and information indicating the size of the screen display area. If the information indicating the size of the screen display area indicates the size of the screen display area, the conversion unit 553 may refer to the standard information 541 to identify information such as height and width corresponding to the size. The conversion unit 553 can calculate conversion parameters for converting to the shape indicated by the identified height and width.

[0042] The character recognition unit 554 performs character recognition processing, such as OCR, based on the converted image data. The character recognition unit 554 may perform character recognition processing using general methods. The character recognition unit 554 also stores the image data after character recognition processing as processed image data information 543 in the storage unit 540.

[0043] As described above, the character recognition unit 554 performs character recognition processing on the converted image data generated by the conversion unit 553. By performing character recognition processing on the converted image data, the accuracy of the character recognition processing can be improved in cases where the screen display unit of the operated device 200 cannot be captured from directly in front, such as when the imaging position of the imaging device 300 is misaligned.

[0044] The display unit 555 displays the processed image data and the like on the screen display unit 520. The display unit 555 may also display various other information on the screen display unit 520 besides the processed image data.

[0045] The operation information transmission unit 556 transmits operation information to the mouse / keyboard receiver 400 in response to operations performed on the operation input unit 510. For example, when the display unit 555 is displaying processed image data or the like on the screen display unit 520, the operation information transmission unit 556 transmits operation information to the mouse / keyboard receiver 400 in response to operations performed on the operation input unit 510. The operation information transmission unit 556 may be implemented using known technologies.

[0046] As described above, the remote control system 100 may include a mobile terminal 600. The mobile terminal 600 may be a general smartphone with tethering capabilities. However, if the imaging device 300 or the mouse / keyboard receiver 400 have wireless communication capabilities, the remote control system 100 does not need to include the mobile terminal 600.

[0047] The above is an example of the configuration of the remote control system 100. According to the above configuration, as shown in Figure 5, the remote control device 500 of the remote control system 100 performs character recognition processing after converting the shape of the screen display part in the image data. As a result, character recognition processing can be performed with high accuracy even when the imaging position of the imaging device 300 is shifted.

[0048] Furthermore, as shown in Figure 6, the remote control system 100 described in this embodiment allows for the transmission of operation information corresponding to operations on the operation input unit 510 to the mouse / keyboard receiver 400 while the processed image data is displayed on the screen display unit 520. As a result, convenience can be improved when performing remote operations based on image data.

[0049] Next, we will explain an example of the operation of the remote control device 500 with reference to Figures 7 and 8. First, we will explain an example of the operation of the remote control device 500 when displaying information on the screen, with reference to Figure 7.

[0050] Figure 7 is a flowchart illustrating an example of the operation of the remote control device 500. Referring to Figure 7, the receiving unit 551 receives image data from the imaging device 300 (step S101). The receiving unit 551 may also receive information from the imaging device 300 indicating the date and time and location where the image data was acquired, along with the image data. In addition, the receiving unit 551 can acquire information indicating the size of the screen display area of ​​the controlled device 200 from a mobile terminal 600 or other external device.

[0051] The extraction unit 552 extracts the screen display unit of the operated device 200, which is the subject, from the image data contained in the image data information 542 (step S102). The extraction unit 552 may extract the screen display unit from the image data using known means.

[0052] The conversion unit 553 performs homography conversion processing based on the extraction results by the extraction unit 552 and information indicating the size of the screen display area (step S103). For example, the conversion unit 553 calculates conversion parameters necessary when converting the shape of the screen display area in the image data to a shape corresponding to the size of the screen display area. The conversion unit 553 also generates converted image data in which the shape of the screen display area has been converted by performing calculations using the calculated conversion parameters.

[0053] The character recognition unit 554 performs character recognition processing, such as OCR, based on the converted image data (step S104). The character recognition unit 554 may perform character recognition processing using a general method.

[0054] The display unit 555 displays the processed image data, which is the image data after processing by the character recognition unit 554, on the screen display unit 520 (step S105).

[0055] The above is an example of the operation of the remote control device 500 when displaying information on the screen display unit 520. Next, with reference to Figure 8, an example of the operation of the remote control device 500 when transmitting operation information will be described.

[0056] Figure 8 is a flowchart showing an example of the operation of the remote control device 500. Referring to Figure 8, the operation information transmission unit 556 acquires operation information corresponding to the operation of the operation input unit 510 while the display unit 555 is displaying the processed image data on the screen display unit 520 (step S201).

[0057] Furthermore, the operation information transmission unit 556 transmits the acquired operation information to the mouse / keyboard receiver 400 (step S202). Subsequently, the mouse / keyboard receiver 400 outputs the received operation information to the device to be operated 200, so that the operation on the mouse or keyboard is recognized by the device to be operated 200.

[0058] The above is an example of the operation of the remote control device 500 when transmitting operation information.

[0059] Thus, the remote control device 500 includes a conversion unit 553, a character recognition unit 554, and a display unit 555. With this configuration, the character recognition unit 554 can perform character recognition processing using image data that has been converted by the conversion unit 553. The display unit 555 can also display the processed image data on the screen display unit 520. As a result, even if the imaging position of the imaging device 300 is slightly off, character recognition processing can be performed with high accuracy, and convenient remote control can be easily realized.

[0060] Furthermore, the remote control device 500 has an operation information transmission unit 556. With this configuration, the operation information transmission unit 556 can transmit operation information corresponding to operations on the operation input unit 510 to the mouse / keyboard receiver 400 while the processed image data is displayed on the screen display unit 520. As a result, convenience can be improved when performing remote control based on image data.

[0061] Note that the configuration of the remote control device 500 and other components is not limited to the example given in this embodiment. For example, referring to Figure 9, the arithmetic processing unit 550 can implement the resolution correction instruction unit 557 in addition to the above-mentioned processing units by reading and executing the program 544.

[0062] The resolution correction instruction unit 557 can instruct the imaging device 300 to change the resolution if it determines that moiré patterns are occurring based on image data information 542 or the like, or in response to operations on the remote control device 500. When the resolution correction instruction unit 557 instructs the imaging device 300 to change the resolution, the resolution in the imaging device 300 is changed, enabling more comfortable remote operation. The resolution correction instruction unit 557 may determine whether or not moiré patterns are occurring by any method of its choosing.

[0063] [Second Embodiment] Next, a second embodiment of the present disclosure will be described with reference to Figures 10 and 11. In the second embodiment of the present disclosure, an overview of the configuration of the remote control device 700, which is an information processing device, will be described.

[0064] Figure 10 shows an example of the hardware configuration of the remote control device 700. Referring to Figure 10, the remote control device 700 has the following hardware configuration as an example. ·CPU (Central Processing Unit) 701 (computing unit) ROM (Read Only Memory) 702 (Storage Device) • RAM (Random Access Memory) 703 (storage device) • Program group 704 loaded into RAM703 • Storage device 705 for storing the program group 704 • Drive device 706 for reading and writing to recording medium 710 outside the information processing device. • Communication interface 707 that connects to the communication network 711 outside the information processing device. • Input / output interface 708 for data input / output. • Bus 709 connecting each component

[0065] Furthermore, the remote control device 700 can realize the functions of the receiving unit 721, conversion unit 722, character recognition unit 723, and display unit 724 shown in Figure 11 by having the CPU 701 acquire the program group 704 and execute it. The program group 704 is stored in advance in a storage device 705 or ROM 702, for example, and the CPU 701 loads it into RAM 703 or the like and executes it as needed. Alternatively, the program group 704 may be supplied to the CPU 701 via a communication network 711, or it may be stored in advance in a recording medium 710, and the drive device 706 may read the program and supply it to the CPU 701.

[0066] Figure 10 shows an example of the hardware configuration of the remote control device 700. The hardware configuration of the remote control device 700 is not limited to the case described above. For example, the remote control device 700 may consist of only a part of the configuration described above, such as not having a drive device 706.

[0067] The receiving unit 721 receives image data in which at least the screen display unit of the device being operated is the subject.

[0068] The conversion unit 722 performs a predetermined conversion process based on information indicating the size of the screen display unit of the device being operated and image data.

[0069] The character recognition unit 723 performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit 722.

[0070] The display unit 724 displays the processed image data after processing by the character recognition unit 723.

[0071] Thus, the remote control device 700 includes a conversion unit 722, a character recognition unit 723, and a display unit 724. With this configuration, the character recognition unit 723 of the remote control device 700 can perform character recognition processing using image data that has been converted by the conversion unit 722. The display unit 724 can also display the image data after processing by the character recognition unit 723. As a result, even if the imaging position when capturing the image data is slightly off, character recognition processing can be performed with high accuracy, and convenient remote control can be easily realized.

[0072] Furthermore, the information processing device such as the remote control device 700 described above can be realized by incorporating a predetermined program into the information processing device. Specifically, another form of the present invention is a program for the information processing device to implement the following processes: receiving image data with at least the screen display unit of the device to be controlled as the subject; performing a predetermined conversion process based on information indicating the size of the screen display unit of the device to be controlled and the image data; performing character recognition processing on the converted image data generated as a result of the conversion process; and displaying the processed image data after the character recognition processing.

[0073] Furthermore, the remote control method performed by the information processing device described above involves the information processing device receiving image data with at least the screen display unit of the device to be controlled as its subject, performing a predetermined conversion process based on information indicating the size of the screen display unit of the device to be controlled and the image data, performing character recognition processing on the converted image data generated as a result of the conversion process, and displaying the processed image data after the character recognition process.

[0074] Even an invention such as a program (or recording medium) or a remote control method having the above-described configuration can achieve the same effects and advantages as the remote control device 700 described above, and thus achieve the objectives of the present invention described above. Furthermore, a remote control system having a remote control device 700 having the above-described configuration and an imaging device that acquires image data of a screen display unit of a device to be controlled and transmits the acquired image data to the remote control device can also achieve the same objectives.

[0075] <Note> Some or all of the above embodiments may also be described as follows. The outline of the remote control device and the like in the present invention will be described below. However, the present invention is not limited to the following configuration.

[0076] (Note 1) A receiving unit that receives image data in which at least the screen display unit of the operated device is the subject, A conversion unit that performs a predetermined conversion process based on information indicating the size of the screen display unit of the operated device and the image data, A character recognition unit performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit, A display unit that displays the processed image data after processing by the character recognition unit, has Remote control device. (Note 2) The remote control device described in Appendix 1, The conversion unit calculates conversion parameters necessary for converting the shape of the screen display unit in the image data to a shape corresponding to the size of the screen display unit, and performs the conversion process using the calculated conversion parameters. Remote control device. (Note 3) A remote control device as described in Appendix 1 or Appendix 2, The conversion unit calculates a homography transformation matrix as the conversion parameter, which is necessary when converting the shape of the screen display unit in the image data to a shape corresponding to the size of the screen display unit. Remote control device. (Note 4) A remote control device described in any one of the items from Appendix 1 to Appendix 3, The conversion unit identifies the height and width of the screen display unit based on standard information indicating the size of the screen display unit, and performs the conversion process using the identified height and width. Remote control device. (Note 5) A remote control device described in any one of the items from Appendix 1 to Appendix 4, It has an operation information transmission unit that acquires operation information in response to operations performed on the operation input unit and transmits the acquired operation information to an external device. Remote control device. (Note 6) A remote control device described in any one of the items from Appendix 1 to Appendix 5, The system includes an instruction unit that transmits an instruction to the imaging device that acquires the aforementioned image data to change the resolution. Remote control device. (Note 7) Information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. Remote control method. (Note 8) In an information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. A program to perform the processing. (Note 9) A remote control device comprising: a receiving unit that receives image data with at least the screen display unit of a device to be operated as its subject; a conversion unit that performs a predetermined conversion process based on information indicating the size of the screen display unit of the device to be operated and the image data; a character recognition unit that performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit; and a display unit that displays the processed image data after processing by the character recognition unit, An imaging device that acquires image data with the screen display unit of the operated device as the subject, and transmits the acquired image data to the remote control device, has Remote control system. (Note 10) The remote control system described in Appendix 9, The remote control device has an operation information transmission unit that acquires operation information corresponding to an operation input unit and transmits the acquired operation information to an external device. The remote control system includes a receiver that receives the operation information transmitted by the operation information transmission unit and outputs it to the device to be controlled. Remote control system.

[0077] The programs described in each of the above embodiments and appendices are stored in a storage device or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, optical disk, magneto-optical disk, or semiconductor memory.

[0078] Although the present invention has been described above with reference to the embodiments described above, the present invention is not limited to the embodiments described above. Various modifications to the structure and details of the present invention can be made within the scope of the present invention as can be understood by those skilled in the art. [Explanation of symbols]

[0079] 100 Remote Control Systems 200 Operated device 300 Imaging devices 400 Mouse and Keyboard Receivers 500 Remote Control Devices 510 Operation Input Section 520 Screen display section 530 Communication I / F Section 540 Storage section 541 Standard Information 542 Image data information 543 Processed image data information 544 Programs 550 Arithmetic Processing Unit 551 Receiving Unit 552 Extraction part 553 Conversion Unit 554 Character recognition section 555 Display section 556 Operation Information Transmission Unit 557 Resolution correction instruction section 600 mobile devices 700 Remote control device 701 CPU 702 ROM 703 RAM 704 Program Groups 705 Storage device 706 Drive unit 707 Communication Interface 708 Input / Output Interface 709 Bus 710 Recording media 711 Communications Network 721 Receiving Unit 722 Conversion Unit 723 Character recognition section 724 Display section

Claims

1. A receiving unit that receives image data in which at least the screen display unit of the operated device is the subject, A conversion unit that performs a predetermined conversion process based on information indicating the size of the screen display unit of the operated device and the image data, A character recognition unit performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit, A display unit that displays the processed image data after processing by the character recognition unit, It has, The system includes an instruction unit that, when it is determined that moiré patterns are occurring based on the aforementioned image data, sends an instruction to the imaging device that acquires the image data to change the resolution. Remote control device.

2. A remote control device according to claim 1, The conversion unit calculates conversion parameters necessary for converting the shape of the screen display unit in the image data to a shape corresponding to the size of the screen display unit, and performs the conversion process using the calculated conversion parameters. Remote control device.

3. A remote control device according to claim 2, The conversion unit calculates a homography transformation matrix as the conversion parameter, which is necessary when converting the shape of the screen display unit in the image data to a shape corresponding to the size of the screen display unit. Remote control device.

4. A remote control device according to any one of claims 1 to 3, The conversion unit identifies the height and width of the screen display unit based on standard information indicating the size of the screen display unit, and performs the conversion process using the identified height and width. Remote control device.

5. A remote control device according to any one of claims 1 to 4, It has an operation information transmission unit that acquires operation information in response to operations performed on the operation input unit and transmits the acquired operation information to an external device. Remote control device.

6. Information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. If it is determined that moiré patterns are occurring based on the aforementioned image data, an instruction is sent to the imaging device that acquires the aforementioned image data to change the resolution. Remote control method.

7. In an information processing device, The system receives image data in which at least the screen display unit of the operated device is the subject. Based on the information indicating the size of the screen display unit of the operated device and the image data, a predetermined conversion process is performed. Character recognition processing is performed on the converted image data generated as a result of the conversion process. Display the processed image data after character recognition processing. If it is determined that moiré patterns are occurring based on the aforementioned image data, an instruction is sent to the imaging device that acquires the aforementioned image data to change the resolution. A program to perform the processing.

8. A remote control device comprising: a receiving unit that receives image data with at least the screen display unit of a device to be operated as the subject; a conversion unit that performs a predetermined conversion process based on information indicating the size of the screen display unit of the device to be operated and the image data; a character recognition unit that performs character recognition processing on the converted image data generated as a result of the conversion process by the conversion unit; a display unit that displays the processed image data after processing by the character recognition unit; and an instruction unit that, when it is determined that moiré patterns have occurred based on the image data, sends an instruction to the imaging device that acquires the image data to change the resolution. An imaging device that acquires image data with the screen display unit of the operated device as the subject, and transmits the acquired image data to the remote control device, has Remote control system.

9. A remote control system according to claim 8, The remote control device has an operation information transmission unit that acquires operation information corresponding to an operation input unit and transmits the acquired operation information to an external device. The remote control system includes a receiver that receives the operation information transmitted by the operation information transmission unit and outputs it to the device to be controlled. Remote control system.

Images