Information processing device, information processing method and storage medium

By setting a writing area in the information processing device and responding to its changes, the problem of inappropriate recording of writing information in the prior art is solved, efficient information storage and integrity are achieved, and natural changes in writing behavior are adapted to.

CN114514499BActive Publication Date: 2024-11-26SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080070912.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-10-17
Filing Date
2020-10-02
Publication Date
2024-11-26
Estimated Expiration
2040-10-02

AI Technical Summary

Technical Problem

It is difficult for existing techniques to properly record information written on the writing object, especially during the addition and erasure of writing information, which may lead to unnecessary recording or loss of information.

Method used

By setting a writing area in the information processing device and controlling the recording of the writing image in response to changes in the area, the area setting unit and the recording control unit work together to ensure that recording is performed only when the writing information is effectively added or erased, and avoiding Unnecessary data storage.

Benefits of technology

It realizes efficient recording and storage of writing information, ensures the integrity and accuracy of information, avoids unnecessary recording and data loss, and adapts to the natural changes in human writing behavior.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114514499B_ABST
    Figure CN114514499B_ABST
Patent Text Reader

Abstract

The present disclosure relates to an information processing device, an information processing method, and a program that enable information to be written on a writing object to be properly recorded. An area setting unit sets a writing area including writing information in a writing image showing a writing object on which writing information is written, and a recording control unit controls recording of the writing image according to a change in the writing area. The present disclosure can be applied to, for example, a blackboard recording system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program that allow information written on a writing object to be appropriately recorded. Background Art

[0002] Generally speaking, meetings and lectures are often conducted while writing and erasing information on a writing object such as a blackboard or a whiteboard.

[0003] It is very convenient to record the information written on the writing object so that the information can be consulted later. On the other hand, constantly recording information will waste storage capacity. Therefore, a technology for efficiently recording the information written on the writing object is needed.

[0004] For example, Patent Document 1 discloses a technology in which information on a blackboard is imaged by a digital camera while a writer holds (eg, picks up) a blackboard eraser, thereby recording information written on the blackboard before the information on the blackboard is erased.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2005-125628 Summary of the invention

[0008] Problems to be solved by the present invention

[0009] However, in the technology disclosed in Patent Document 1, there is a possibility that information written on a writing object cannot be appropriately recorded.

[0010] The present disclosure has been made in view of such circumstances, and is intended to enable information written on a writing object to be appropriately recorded.

[0011] Solution to the problem

[0012] The present disclosure provides an information processing device including: an area setting unit that sets a writing area including writing information in a writing image showing a writing object having written writing information; and a recording control unit that controls recording of the writing image in response to a change in the writing area.

[0013] The present disclosure provides an information processing method executed by an information processing device or a program for causing a computer to execute processing, including: setting a writing area including writing information in a writing image including writing information written on a writing object; and controlling the recording of the writing image in response to a change in the writing area.

[0014] In the present disclosure, a writing area including writing information is set in a writing image including writing information written on a writing object, and recording of the writing image is controlled in response to a change of the writing area. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a diagram illustrating an overview of a blackboard recording system according to an embodiment of the present disclosure.

[0016] Figure 2 is a block diagram illustrating a functional configuration example of a blackboard recording system.

[0017] Figure 3 It is a flowchart illustrating the flow of blackboard recording processing.

[0018] Figure 4 is a diagram illustrating an example of a captured image.

[0019] Figure 5 The figure illustrates the extraction of the blackboard writing image.

[0020] Figure 6 The figure illustrates the extraction of the blackboard writing image.

[0021] Figure 7 is a diagram illustrating an example of a blackboard writing image.

[0022] Figure 8 This is a diagram illustrating the settings of the blackboard writing area.

[0023] Fig. 9 is a diagram illustrating recording control in response to a change of a blackboard writing area.

[0024] Fig.10 is a flowchart illustrating details of the recording control process.

[0025] Fig.11 is a flowchart illustrating details of the recording control process.

[0026] Fig.12 is a diagram illustrating an example of mode transition.

[0027] Fig.13 is a diagram illustrating an example of mode transition.

[0028] Fig.14 is a block diagram illustrating another functional configuration example of the information processing apparatus.

[0029] Fig.15 It is a flowchart illustrating the blackboard recording process.

[0030] Fig.16 : is a diagram illustrating an example of setting the blackboard writing remaining time.

[0031] Fig.17 It is a diagram illustrating shielding of the recording area.

[0032] Fig.18 is a flowchart illustrating details of the recording control process.

[0033] Fig.19 is a diagram illustrating an example of mode transition.

[0034] Fig. 20 It is a diagram illustrating the record based on the remaining time of the blackboard writing.

[0035] Fig.21 is a flowchart illustrating details of the recording control process.

[0036] Fig. 22 is a diagram illustrating an example of mode transition.

[0037] Fig.23 is a block diagram illustrating a configuration example of computer hardware. DETAILED DESCRIPTION

[0038] A mode for carrying out the present disclosure (hereinafter referred to as an “embodiment”) will be described below. Note that the description will be made in the following order.

[0039] 1. Overview of the blackboard recording system

[0040] 2. Configuration example of blackboard recording system

[0041] 3. Process of blackboard recording

[0042] 3-1. Details of logging control by mode

[0043] 3-2. Example of mode transition

[0044] 4. Another configuration example of the blackboard recording system

[0045] 5. Process of blackboard recording

[0046] 5-1. Masking of recording area

[0047] 5-2. Record control based on the remaining time of the blackboard writing

[0048] 6. Computer Configuration Example

[0049] <1. Overview of the blackboard recording system>

[0050] Figure 1 : is a diagram illustrating an overview of the blackboard recording system according to the present embodiment.

[0051] like Figure 1As shown in , the blackboard recording system 1 includes an information processing apparatus 10 and an input device 20 connected to the information processing apparatus 10 .

[0052] The writing object 2 is an object on which visual information (hereinafter referred to as writing information 3 ) such as dots, lines, characters, sentences, mathematical formulas, symbols, pictures, figures, or images is written.

[0053] The writing object 2 is composed of a blackboard, a whiteboard, an electronic paper, a touch panel, etc. The written information 3 is written on the writing object 2 with chalk, a marker, a stylus, a finger, etc., and is erased from the writing object 2 with a blackboard eraser, an eraser, etc.

[0054] The writer 4 is a person who operates the writing object 2 .

[0055] The writer 4 writes written information 3 on the writing object 2, and erases the written information 3 written on the writing object 2. For example, in a lecture or a meeting, the writer 4 gives a description about the written information 3 while pointing to the written information 3 written on the writing object 2 with a hand, a laser pen, a pointer, or the like.

[0056] The input device 20 is a device for inputting information about a physical space in which the input device 20 is installed. The input device 20 includes, for example, an imaging device and an audio input device.

[0057] The imaging device includes a lens system consisting of an imaging lens, an aperture, a zoom lens, a focusing lens, etc., a driving system that enables the lens system to perform a focusing operation or a zooming operation, and an image sensor that performs photoelectric conversion on the imaging light obtained by the lens system to generate an imaging signal.

[0058] The audio input device includes a microphone that collects ambient sound, a microphone amplifier circuit that amplifies an audio signal obtained by the microphone, an A / D converter, and a signal processing circuit such as a noise canceller.

[0059] The input device 20 outputs image data converted into digital signals and audio data at the time of imaging.

[0060] The input device 20 can capture an image of an object as an imaging target in a physical space. In the present embodiment, the input device 20 captures an image of a writing object 2 as an imaging target on which writing information 3 is written, and outputs the imaging time and the acquired image to the information processing apparatus 10 in association with each other.

[0061] Note that the image acquired by imaging may include other objects than the writing object 2. In that case, the input device 20 outputs an image extracted from the image acquired by imaging to the information processing apparatus 10. For example, an image obtained by extracting a region in which the writing object 2 is reflected from the image acquired by imaging is output to the information processing apparatus 10.

[0062] In addition, a function as the input device 20 may be included in the writing object 2. For example, the input device 20 and the writing object 2 are configured as an electronic blackboard that acquires and outputs an image by scanning the state of the writing object 2. In this case, the input device 20 acquires the image of the writing object 2, and outputs the acquired time and the acquired image to the information processing apparatus 10 in association with each other.

[0063] The image input by the input device 20 may be a moving image. The moving image is data including a plurality of still images (image data) and a reproduction time (corresponding to the time of imaging) of each still image. In the case of reproducing a moving image, the still images are sequentially reproduced in the order of the reproduction time. Still images constituting the moving image are also referred to as frames. The number of frames per unit time is also referred to as a frame rate, and is expressed in frames per second (FPS).

[0064] Furthermore, the image input by the input device 20 may be a still image. In that case, the input device 20 sequentially inputs the images. In this specification, the number of images input per unit time is also referred to as a frame rate following a moving image.

[0065] Hereinafter, the image of the writing object 2 on which the writing information 3 input by the input device 20 is written is referred to as a writing image or a blackboard image. It is assumed that the blackboard image is a still image. In the case where the image input by the input device 20 is a moving image, the blackboard image is a still image constituting the moving image.

[0066] The information processing device 10 selectively records the blackboard writing image to be recorded among the blackboard writing images input by the input device 20. As used herein, recording means storing information in a storage medium. The information processing device 10 monitors the blackboard writing image and records the blackboard writing image when a change in the writing information 3 written on the writing object 2 is detected. The blackboard writing image selectively recorded based on the change in the writing information is also referred to as a recorded image hereinafter.

[0067] The change of the writing information 3 may be the erasure of the writing information 3. In that case, the information processing device 10 records the blackboard image before the start of erasure as the recorded image. Usually, before the erasure of the writing information 3 starts, the maximum amount of writing information 3 is written on the writing object 2. Therefore, it is possible to selectively record the blackboard image before the erasure starts, thereby suppressing unnecessary recording and efficiently recording the blackboard image.

[0068] The change of the written information 3 may be the writing (i.e., adding) of the written information 3. In that case, the information processing device 10 records the blackboard image as a recorded image after the adding is completed. Usually, after the adding of the written information 3 is completed, the maximum amount of the written information 3 is written on the writing object. Therefore, it is possible to selectively record the blackboard image after the adding is completed, thereby suppressing unnecessary recording and efficiently recording the blackboard image.

[0069] <2. Configuration example of blackboard recording system>

[0070] Figure 2 : is a block diagram illustrating a functional configuration example of the blackboard recording system 1 .

[0071] Figure 2 The blackboard recording system 1 includes an information processing apparatus 10 , an input device 20 and an output device 30 .

[0072] As described above, the input device 20 sequentially inputs the blackboard writing images and outputs each blackboard writing image to the information processing apparatus 10 .

[0073] The output device 30 is a device that outputs information under the control of the information processing apparatus 10 .

[0074] The output device 30 is constituted by a display device such as a liquid crystal display device, a plasma display device, an organic electroluminescence (EL) display device, a laser projector, a light emitting diode (LED) projector, and a lamp.

[0075] The output device 30 displays a blackboard writing image (recorded image) so that, for example, students taking a class in a classroom and an audience watching the lecture in a distant classroom can see the writing information 3 written on the writing object 2 in real time.

[0076] The information processing apparatus 10 is a device that controls the operation of the entire blackboard recording system 1. The information processing apparatus 10 is constituted by an optional device such as a personal computer (PC), a smartphone, a tablet terminal, or dedicated equipment.

[0077] like Figure 2 As shown in , the information processing device 10 includes an image acquisition unit 51, a blackboard image extraction unit 52, a blackboard area setting unit 53, a recording control unit 54, a storage unit 55 and an output control unit 56.

[0078] The image acquisition unit 51 acquires an image in units of frames input by the input device 20 , for example, and supplies the image to the blackboard-writing image extraction unit 52 .

[0079] The blackboard writing image extraction unit 52 extracts the area where the writing object 2 is reflected as a blackboard writing image from the image from the image acquisition unit 51. The extracted blackboard writing image is supplied to the blackboard writing area setting unit 53.

[0080] The blackboard writing area setting unit 53 sets a writing area in a blackboard writing image (writing image) showing a writing object 2, and the writing area is an area including writing information 3. The writing area is an area indicating the proportion of the range where the writing information 3 is written to the entire writing object 2. The writing area is also referred to as the blackboard writing area hereinafter. The blackboard writing image in which the blackboard writing area has been set is supplied to the recording control unit 54.

[0081] The recording control unit 54 controls the recording of the blackboard writing image in the storage unit 55 in response to the change of the writing area in the blackboard writing image in units of frames, specifically, depending on whether the blackboard writing area increases or decreases.

[0082] The recording control unit 54 includes a mode setting unit 61. The mode setting unit 61 sets a blackboard status mode indicating a change status of the writing information 3 in the blackboard image (writing object 2). The blackboard status mode includes a "writing on the blackboard" mode, which indicates a status in which the writing information 3 is being written (added) on the writing object 2, and an "erasing from the blackboard" mode, which indicates a status in which the writing information 3 is erased from the writing object 2.

[0083] The recording control unit 54 controls the recording of the blackboard writing image in the storage unit 55 according to the blackboard writing state mode set by the mode setting unit 61 .

[0084] The storage unit 55 records the blackboard writing image as a recorded image under the control of the recording control unit 54 .

[0085] The output control unit 56 generates output information based on the blackboard image stored in the storage unit 55 , and causes the output device 30 to output the output information.

[0086] <3. Process of blackboard recording>

[0087] Next, we will refer to Figure 3 The flowchart in FIG. 1 describes the process of the blackboard recording process performed by the blackboard recording system 1. For each frame of the image input from the input device 20, Figure 3 in the processing.

[0088] In step S11 , the image acquisition unit 51 acquires an image input from the input device 20 .

[0089] Figure 4 An example of a captured image input from the input device 20 is illustrated. Figure 4 In the captured image 80 shown in FIG. 8 , the writing object 2 on which the written information 3 is written and the writer 4 are reflected. The captured image 80 may be one frame of a moving image, or may be a still image.

[0090] In the case where the writing object 2 appears small or the adjustment of the white balance is inappropriate in the captured image 80, there is a possibility that an error occurs in the determination of the addition or erasure of the writing information 3. Therefore, in the captured image 80, the writing object 2 is made to appear sufficiently large, and the writing information 3 written on the writing object 2 is made to appear in a color close to the color depth seen by the naked eye.

[0091] In step S12, the blackboard writing image extraction unit 52 extracts the blackboard writing image from the captured image acquired by the image acquisition unit 51. The extraction of the blackboard writing image may be performed automatically or manually.

[0092] In the case of manually extracting blackboard images, such as Figure 5 As shown in , four points c1, c2, c3 and c4 at four corners of the writing object 2 are specified in the captured image 80 so that the area of ​​the writing object 2 is specified, and the blackboard image 90 is extracted through the projective transformation of the four points c1 to c4.

[0093] Next, the shape of the writer 4 is recognized in the blackboard image 90, and the area where the writer 4 is reflected is excluded, so that the blackboard image 90 in which only the writing object 2 is reflected is obtained, as shown in FIG. Figure 6 As shown in . As a technique for removing the area reflecting the writer 4 from the blackboard image 90, a technique such as moving object removal is used. Alternatively, another technique may be used.

[0094] Furthermore, the blackboard image 90 from which the writer 4 has been removed is subjected to a binarization process, wherein pixels in which the writing information 3 is written in the writing object 2 are converted to white (e.g., 1), and the remaining pixels are converted to black (e.g., 0). By this arrangement, a binary image is obtained as shown in FIG. Figure 7 The binary image composed of black pixels and white pixels shown in FIG. 1 is used as the blackboard image 100 .

[0095] In step S13 , the blackboard writing area setting unit 53 sets the blackboard writing area in the blackboard writing image extracted by the blackboard writing image extraction unit 52 .

[0096] Specifically, the blackboard writing area setting unit 53 sets the blackboard writing area by performing expansion processing on the written information 3 in the blackboard writing image. Also, the blackboard writing area setting unit 53 performs resolution reducing processing on the blackboard writing image in which the blackboard writing area has been set.

[0097] For example, suppose Figure 8 The blackboard writing image 110 shown on the left has been extracted by the blackboard writing image extraction unit 52. The blackboard writing image 110 is a binary image having a resolution of 1920×1080 pixels and occupying approximately 5% of the white pixel area of ​​the entire blackboard writing image 110.

[0098] In the case where there is at least one white pixel near the pixel of interest in the blackboard image 110 as described above, the pixel of interest is replaced with a white pixel through the expansion process. Furthermore, the blackboard image 110 that has been subjected to the expansion process is subjected to a resolution reduction process.

[0099] With this arrangement, Figure 8 The blackboard image 110 shown on the left is converted into Figure 8 The blackboard image 120 shown on the right is a binary image having a resolution of 320×180 pixels and white pixels occupying approximately 22% of the entire blackboard image 120 .

[0100] A blackboard writing image in which a blackboard writing area has been set as in the blackboard writing image 120 is hereinafter also referred to as a blackboard writing area image.

[0101] In the blackboard writing area image, the white pixel area is the blackboard writing area indicating the proportion of the range of the writing information 3 written to the writing object 2. The size of the blackboard writing area is an indicator showing how much of the writing object 2 is used for writing.

[0102] Specifically, in Figure 8 In the example, the blackboard image 110 has an area of ​​white pixels that occupies approximately 5% of the entire blackboard image 110 , while the blackboard image 120 has an area of ​​white pixels that occupies approximately 22% of the entire blackboard image 120 .

[0103] In the case where recording of a blackboard writing image is controlled only in response to a change in writing information, whether or not to record a blackboard writing image is determined based on the amount of increase and decrease of lines and dots written with chalk or a marker.

[0104] However, even in the case where the written information occupies the same area in the blackboard image, whether the blackboard image is recorded may vary depending on the density of the multiple characters being written or the density of the center line of the characters themselves, and may deviate from human perception. For example, in the case where the density of the written characters is low, even if those characters have been erased, the blackboard image may not be recorded.

[0105] Therefore, by controlling recording in response to a change in the blackboard writing area set in the blackboard writing image, it is possible to determine whether the blackboard writing image is to be recorded in a state closer to human feeling.

[0106] Note that instead of performing expansion processing on the writing information in the blackboard image for setting the blackboard area, character recognition may be performed in the blackboard image, and the character area in which the character recognition has been performed may be set as the blackboard area.

[0107] Now, return to Figure 3 As described in the flowchart in , in step S14, the record control unit 54 performs record control processing according to the blackboard writing state mode.

[0108] First, the mode setting unit 61 of the record control unit 54 sets the blackboard writing state mode in response to the change of the blackboard writing area. Specifically, if the blackboard writing area has increased, the mode setting unit 61 sets the "writing on the blackboard" mode to the blackboard writing state mode. On the other hand, if the blackboard writing area has decreased, the mode setting unit 61 sets the "erasing from the board" mode to the blackboard writing state mode.

[0109] Then, the record control unit 54 controls the recording of the blackboard image in the storage unit 55 based on the blackboard status mode that has been set. Specifically, in the case where the "Writing on the Blackboard" mode has been set, since the writing information has been added to the blackboard image, the record control unit 54 updates the record candidate image as a candidate for the recorded image. On the other hand, in the case where the "Erase from the Blackboard" mode has been set, since the writing information has been erased from the blackboard image, the record control unit 54 records the record candidate image at that time as the recorded image.

[0110] Fig. 9 is a diagram illustrating recording control in response to a change of a blackboard writing area in a blackboard writing image.

[0111] Fig. 9 The blackboard writing image, the blackboard writing area image, and the recording candidate image at each of times t1 to t4 are illustrated.

[0112] First, in the blackboard image at time t1, "AB" is written as writing information, and a blackboard area image in which the range of writing "AB" has been set to white pixels is obtained. The record candidate image at time t1 is a record candidate image updated with the blackboard image in which "AB" is written.

[0113] When "C" is written as writing information in addition to "AB" at time t2, a blackboard writing area image is obtained in which the range where "ABC" is written has been set to white pixels. At this time, since the blackboard writing area has increased, the "writing on the board" mode is set, and the candidate image is updated with the blackboard writing image in which "ABC" is written.

[0114] When "ABC" as the written information is completely erased from the blackboard image at time t3, a blackboard area image in which all pixels have been set to black pixels is obtained. At this time, since the blackboard area has been reduced, the "erase from the board" mode is set, and the record candidate image in which "ABC" is written is recorded as the recorded image.

[0115] Moreover, when "DEF" is written as writing information in the blackboard image at time t4, a blackboard area image in which the range in which "DEF" is written is set to white pixels is obtained. At this time, since the writing area has increased, the "writing on the board" mode is set, and the candidate image is updated with the blackboard image in which "DEF" is written.

[0116] In this way, the blackboard writing state mode is set in response to the change of the blackboard writing area, and the recording of the blackboard writing image is controlled based on the blackboard writing state mode that has been set.

[0117] According to the above processing, the recording of the blackboard image is controlled in response to the change of the blackboard area set in the blackboard image, rather than in response to the change of the written information in the blackboard image. Through this arrangement, whether the blackboard image will be recorded is determined in a state closer to human feeling regardless of the density of the written characters, and this makes it possible to appropriately record the information written on the writing object.

[0118] (3-1. Details of recording control by mode)

[0119] Here, we will refer to Fig.10 and 11 Flowchart description in Figure 3 In step S14, the details of the recording control processing are performed according to the blackboard writing status mode. Fig.10 and 11 The processing in is also performed on the blackboard image for each frame.

[0120] In step S31, the record control unit 54 determines whether the blackboard writing state mode is the "writing on the blackboard" mode. As will be described later, the blackboard writing state mode of the current frame is determined based on the change of the blackboard writing area in the previous frame. Note that the blackboard writing state mode of the first frame is set to the "erasing from the board" mode.

[0121] If it is determined that the blackboard writing state mode is the "writing on the blackboard" mode, the process proceeds to step S32.

[0122] In step S32, the record control unit 54 determines whether the blackboard area has increased by one pixel or more from the current value. Assume that the size of the blackboard area is stored in pixels. The current value of the blackboard area is maintained in a predetermined storage area and is appropriately updated for each frame of the blackboard image.

[0123] If it is determined that the blackboard writing area has increased by one pixel or more from the current value, that is, if writing information has been added on the writing object 2, the process proceeds to step S33.

[0124] In step S33, the record control unit 54 updates the record candidate image with the blackboard image at that time. The record candidate image is held in a predetermined storage area, and when the blackboard area increases, for example, the record candidate image is updated with the blackboard image at that time.

[0125] When the record candidate image has been updated, in step S34, the record control unit 54 updates the current value and the maximum value of the blackboard area with the number of pixels of the blackboard area (the size of the blackboard area) at that time. The maximum value of the blackboard area is also maintained in a predetermined storage area and is appropriately updated for each frame of the blackboard image.

[0126] Then, in step S35 , the recording control unit 54 maintains the “writing on the board” mode as the blackboard writing state mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0127] Here, the processing flow of steps S31 to S35 is appropriately referred to as flow F1.

[0128] Now, if it is determined in step S32 that the blackboard writing area has not increased by one pixel or more from the current value, the processing proceeds to step S36.

[0129] In step S36 , the record control unit 54 determines whether the blackboard writing area has decreased by 5% or more from the maximum value.

[0130] If it is determined that the blackboard writing area has been reduced by 5% or more from the maximum value, that is, if a certain amount or more of written information has been erased from the writing object 2, the process proceeds to step S37.

[0131] In step S37 , the record control unit 54 records the record candidate image at that time as a recorded image.

[0132] When the recorded image has been recorded, in step S38, the recording control unit 54 updates the minimum value of the blackboard area with the number of pixels of the blackboard area at that time (the size of the blackboard area). The minimum value of the blackboard area is also maintained in a predetermined storage area and is appropriately updated for each frame of the blackboard image.

[0133] Then, in step S39 , the recording control unit 54 transitions the blackboard writing state mode to the “erase from board” mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0134] Here, the processing flow of steps S31, S32, S36 to S39 is appropriately referred to as flow F2.

[0135] Now, if it is determined in step S36 that the blackboard writing area has not decreased by 5% or more from the maximum value, the process proceeds to step S40.

[0136] In step S40 , the record control unit 54 updates the current value of the blackboard writing area with the number of pixels of the blackboard writing area at that time (the size of the blackboard writing area).

[0137] Then, in step S41 , the recording control unit 54 maintains the “writing on the board” mode as the blackboard writing state mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0138] Here, the processing flow of steps S31 , S32 , S36 , S40 , and S41 is appropriately referred to as flow F3 .

[0139] By the way, if it is determined in step S31 that the blackboard writing state mode is not the "writing on the blackboard" mode, that is, if the blackboard writing state mode is the "erasing from the blackboard" mode, then the process proceeds to step S41. Fig.11 Step S42 in .

[0140] In step S42 , the record control unit 54 determines whether the blackboard writing area has decreased by one pixel or more from the minimum value.

[0141] If it is determined that the blackboard writing area has decreased by one pixel or more from the minimum value, that is, if the writing information has been erased from the writing object 2, the process proceeds to step S43.

[0142] In step S43 , the record control unit 54 updates the minimum value of the blackboard writing area with the number of pixels of the blackboard writing area at that time (the size of the blackboard writing area).

[0143] Then, in step S44 , the recording control unit 54 maintains the “erase from board” mode as the blackboard writing state mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0144] Here, the processing flow of steps S31, S42 to S44 is appropriately referred to as flow F4.

[0145] Now, if it is determined in step S42 that the blackboard writing area has not decreased by one or more pixels from the minimum value, the process proceeds to step S45.

[0146] In step S45 , the record control unit 54 determines whether the blackboard writing area has increased by 1% or more from the minimum value.

[0147] If it is determined that the blackboard writing area has increased by 1% or more from the minimum value, that is, if a certain amount or more of writing information has been added to the writing object 2, the process proceeds to step S46.

[0148] In step S46 , the record control unit 54 updates the record candidate image with the blackboard image at that time.

[0149] When the recording candidate image has been updated, in step S47 , the recording control unit 54 updates the current value of the blackboard writing area with the number of pixels of the blackboard writing area (the size of the blackboard writing area) at that time.

[0150] Then, in step S48 , the recording control unit 54 transitions the blackboard writing state mode to the “writing on the blackboard” mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0151] Here, the processing flow of steps S31, S42, S45 to S48 is appropriately referred to as flow F5.

[0152] Now, if it is determined in step S45 that the blackboard writing area has not increased by 1% or more from the minimum value, the process proceeds to step S49.

[0153] In step S49 , the recording control unit 54 maintains the “erase from board” mode as the blackboard writing state mode, and the process proceeds to the blackboard writing recording process for the next frame.

[0154] As described above, the recording of the blackboard writing image is controlled according to the blackboard writing state mode.

[0155] (3-2. Example of mode transition)

[0156] Fig.12 The diagram is in reference Fig.10 and 11 The flowchart in the figure describes an example of the mode transition of the blackboard writing state mode in the record control process.

[0157] Fig.12 The change of the writing information in the blackboard writing image at each of the times T1 to T6 is illustrated.

[0158] exist Fig.12 In the example of , the blackboard recording process starts at time T1. The blackboard state mode at this time is set to the "erase from the board" mode.

[0159] At time T2, no written information is written on the blackboard image, and the process of flow F6 in the above-mentioned recording control process is executed. Therefore, the "erase from the board" mode is maintained.

[0160] At time T3, when "A" is written on the blackboard image, the process of flow F5 in the above-mentioned recording control process is executed. Therefore, the recording candidate image is updated with the blackboard image in which "A" is written, and the mode transitions to the "writing on the board" mode.

[0161] At time T4, there is no writing information added from the state at time T3, and the processing of flow F3 in the above-mentioned recording control processing is executed. Therefore, the "writing on the board" mode is maintained.

[0162] At time T5, when "B" is added to the right side of "A" from the state at time T4, the processing of flow F1 in the above-mentioned record control processing is executed. Therefore, the record candidate image is updated with the blackboard image in which "AB" is written, and the "writing on the board" mode is maintained.

[0163] At time T6, when "CDE" is added to the right side of "AB" and "FGHIJ" and "KLMNO" are added below "AB" and "CDE" from the state at time T5, the processing of flow F1 in the above-mentioned record control processing is executed. Therefore, the record candidate image is updated with the blackboard image in which "ABCDE", "FGHIJ" and "KLMNO" are written, and the "writing on the blackboard" mode is maintained.

[0164] Fig.13 It is a graphic Fig.12 FIG. 4 is a diagram of an example of a mode transition after time T6 in FIG.

[0165] Fig.13 Changes in the writing information in the blackboard writing image at each of times T11 to T16 are illustrated.

[0166] At time T11, when Fig.12 When the state at time T6 in the above recording control process erases "ABC", the process of flow F2 in the above recording control process is executed. Therefore, the recording candidate image in which "ABCDE", "FGHIJ" and "KLMNO" are written is recorded as the recorded image 151, and the mode transitions to the "erase from plate" mode.

[0167] At time T12, the information written from the state at time T11 has not been further erased, and the processing of flow F6 in the above-mentioned recording control processing is performed. Therefore, the "erase from plate" mode is maintained.

[0168] At time T13, when "DE" is further erased from the state at time T12, the process of flow F4 in the above-mentioned recording control process is executed. Therefore, the "erase from plate" mode is maintained.

[0169] At time T14, when "P" is written in the place where "A" was written before erasing from the state at time T13, the processing of flow F5 in the above-mentioned record control processing is executed. Therefore, the record candidate image is updated with the blackboard image in which "P", "FGHIJ" and "KLMNO" are written, and the mode transitions to the "writing on the board" mode.

[0170] At time T15, when "QR" is added to the right side of "P" from the state at time T14, the processing of flow F1 in the above-mentioned record control processing is executed. Therefore, the record candidate image is updated with the blackboard image in which "PQR", "FGHIJ" and "KLMNO" are written, and the "writing on the board" mode is maintained.

[0171] Thereafter, at time T16, when "FGHIJ" is erased from the state at time T15, the process of flow F2 in the above-mentioned recording control process is executed. Therefore, the recording candidate image in which "PQR" and "KLMNO" are written is recorded as the recorded image 152, and the mode transitions to the "erase from plate" mode.

[0172] Here, when compared with Fig.13 When the mode transition records recorded image 151 and recorded image 152, although "FGHIJ" and "KLMNO" indicated by the dotted frame in recorded image 152 were once recorded as recorded image 151, "FGHIJ" and "KLMNO" are recorded again.

[0173] That is, in the configuration of the above-described blackboard recording system 1 , a recording candidate image including writing information included in an already recorded recorded image is recorded again as a recorded image, which results in unnecessary recording.

[0174] Therefore, in the following, an example of preventing written information that has already been recorded from being recorded again as described above will be described.

[0175] <4. Another configuration example of the blackboard recording system>

[0176] Fig.14 is a block diagram illustrating another functional configuration example of the blackboard recording system 1 .

[0177] Fig.14 The information processing device 10 and Figure 2 The information processing apparatus 10 in FIG. 1 is different in that the record control unit 54 further includes a pixel information setting unit 161 .

[0178] The pixel information setting unit 161 sets pixel information indicating the state of each pixel in the blackboard writing area in the blackboard writing image (blackboard writing area image).

[0179] For example, for pixels forming a blackboard area corresponding to writing information that has not been erased (not deleted) since the blackboard area image was written, the pixel information setting unit 161 sets the blackboard residual time as pixel information, which is the time elapsed from the writing of the writing information.

[0180] In addition, for pixels forming a blackboard writing area (hereinafter referred to as a recorded area) corresponding to writing information that has been recorded as a recorded image and has not been erased in the blackboard writing area image, the pixel information setting unit 161 sets a recorded flag as pixel information instead of the blackboard writing residual time.

[0181] <5. Process of blackboard recording>

[0182] Next, we will refer to Fig.15 Flowchart description in Fig.14 The process of the blackboard recording process performed by the blackboard recording system 1 in the embodiment of the present invention is also executed for each frame of the image input from the input device 20. Fig.15 in the processing.

[0183] Note that, Fig.15 The processing of steps S111 to S114 in the flowchart in is similar to Figure 3 The processing of steps S11 to S14 in the flowchart in FIG. 1 is described below, and thus a description thereof will be omitted.

[0184] That is, after the recording control process according to the mode has been performed in step S114 , in step S115 , the pixel information setting unit 161 sets pixel information (blackboard writing remaining time) for each pixel of the blackboard writing area in the blackboard writing area image.

[0185] Fig.16 : is a diagram illustrating an example of setting the blackboard writing remaining time.

[0186] Fig.16 The blackboard writing area image at each of times t11 to t15 and the pixel information array in which the blackboard writing undeleted time is set are illustrated.

[0187] In the case where the blackboard writing area image (blackboard writing image) has a resolution of 320×180 pixels, a pixel information array of the same size (320×180 pixels) is prepared, and the blackboard writing residual time is set to a value corresponding to each pixel of the pixel information array.

[0188] exist Fig.16 In the example of , for convenience of explanation, a blackboard writing area image and a pixel information array of 5×5 pixels are illustrated. Hereinafter, for example, a pixel that is the second pixel from the top and the second pixel from the left is referred to as pixel (2, 2).

[0189] In the blackboard writing area image at time t11, there is no white pixel area, and all values ​​of the pixel information array are set to 0.

[0190] When the pixel (2, 2) is set as a white pixel in the blackboard writing area image at time t12, the value corresponding to the pixel (2, 2) is set to 1 in the pixel information array as the blackboard writing residual time.

[0191] When the two pixels (2, 3) and (2, 4) in the blackboard writing area image at time t13 are set as white pixels, in the pixel information array, the values ​​corresponding to the two pixels (2, 3) and (2, 4) are set to 1 as the blackboard writing residual time, and the value corresponding to the pixel (2, 2) is set to 2 as the blackboard writing residual time.

[0192] When the two pixels (3, 3) and (4, 3) in the blackboard area image at time t14 are set as white pixels, in the pixel information array, the values ​​corresponding to the two pixels (3, 3) and (4, 3) are set to 1 as the blackboard remaining time, the values ​​corresponding to the two pixels (2, 3) and (2, 4) are set to 2 as the blackboard remaining time, and the value corresponding to the pixel (2, 2) is set to 3 as the blackboard remaining time.

[0193] Then, at time t15, when no pixel is set to a white pixel from the state at time t14, in the pixel information array, the values ​​corresponding to the two pixels (3, 3) and (4, 3) are set to 2 as the blackboard writing residual time, the values ​​corresponding to the two pixels (2, 3) and (2, 4) are set to 3 as the blackboard writing residual time, and the value corresponding to the pixel (2, 2) is set to 4 as the blackboard writing residual time.

[0194] As described above, the blackboard writing residual time in the blackboard writing area (white pixels) is counted as time goes by.

[0195] (5-1. Masking of recording area)

[0196] Reference Fig.17 An example of masking a recorded area when a recording candidate image is updated in order to prevent once-recorded written information from being recorded again is described.

[0197] Fig.17 A blackboard writing image, a blackboard writing area image, a pixel information array in which a blackboard writing residual time is set, and a recording candidate image at each of times t21 to t24 are illustrated.

[0198] And in Fig.17 In the example, a blackboard writing area image and a 5×5 pixel information array are illustrated.

[0199] In the blackboard image at time t21, "ABC" and "DEF" are written as writing information, and in the blackboard area image, pixels (2, 2), (2, 3), (2, 4), (4, 2), (4, 3), and (4, 4) are set as white pixels. At this time, the candidate image is updated with the blackboard image in which "ABC" and "DEF" are written.

[0200] At this time, in the pixel information array, the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) are respectively set to 6, 5 and 4 as the blackboard writing residual time, and the values ​​corresponding to the three pixels (4, 2), (4, 3) and (4, 4) are respectively set to 3, 2 and 1 as the blackboard writing residual time. That is, in the blackboard writing image, "A" is written earliest, and "F" is written most recently.

[0201] When "ABC" is erased from the state at time t21 in the blackboard image at time t22, pixels (2, 2), (2, 3) and (2, 4) are set as black pixels in the blackboard area image, and a recording candidate image in which "ABC" and "DEF" are written is recorded as a recorded image.

[0202] At this time, in the pixel information array, each of the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) is set to 0 as the blackboard writing residual time, and each of the values ​​corresponding to the three pixels (4, 2), (4, 3) and (4, 4) is set to -1 as the blackboard writing residual time. That is, the recorded flag is set for the three pixels (4, 2), (4, 3) and (4, 4) forming the recorded area.

[0203] In the blackboard image at time t23, when "GHI" is written in the place where "ABC" had been written before erasing from the state at time t22, in the blackboard area image, pixels (2, 2), (2, 3) and (2, 4) are set to white pixels again. At this time, in the pixel information array, the pixels in the recording area (the area where "DEF" is written in the blackboard image) where the recorded flag -1 is set are masked, so only the recording candidate image in which "GHI" is written is recorded as the recorded image.

[0204] At this time, in the pixel information array, the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) are set to 3, 2 and 1 respectively as the blackboard writing residual time.

[0205] When "GH" and "DE" are erased from the state at time t23 in the blackboard image at time t24, in the blackboard area image, pixels (2, 2), (2, 3), (4, 2) and (4, 3) are set to black pixels, and a recording candidate image in which only "GHI" is written is recorded as a recorded image.

[0206] At this time, in the pixel information array, each of the values ​​corresponding to the four pixels (2, 2), (2, 3), (4, 2) and (4, 3) is set to 0 as the blackboard writing remaining time, and the value corresponding to the one pixel (2, 4) in which the written information is not deleted is set to -1 as the blackboard writing remaining time. That is, the recorded flags are set for the two pixels (2, 4) and (4, 4) forming the recorded area.

[0207] Next, we will refer to Fig.18 The flowchart in describes the details of the above-mentioned recording control processing that implements masking of the recorded area.

[0208] Fig.18 The processing flow in steps S131 to S143 in Fig.10 The processing flow of steps S31 to S41 in (flows F1, F2, and F3) is basically similar, but the difference is that the processing of step S133 is added.

[0209] Specifically, in step S132, in the "writing on the board" mode, if it is determined that the writing area has increased by one pixel or more from the current value, for example, if writing information is added after the recording of the recorded image, such as Fig.17 If it is time t23, the processing proceeds to step S133.

[0210] In step S133, the recording control unit 54 masks the recorded area in the blackboard image. The recorded area is the same as the area previously recorded in the blackboard recording process ( Fig.15 ) is set as the pixel of the recorded flag-1 in step S115. With this arrangement, in step S134, the recording candidate image is updated with the blackboard image in which the recorded area (the pixel for which the recorded flag is set) is shielded.

[0211] Fig.19 It is a graphic reference Fig.18 The flowchart described in the figure is an example of the mode transition of the blackboard writing state mode in the recording control process, and is a diagram illustrating Fig.12 FIG. 4 is a diagram of an example of a mode transition after time T6 in FIG.

[0212] Fig.19 Changes in the writing information in the blackboard writing image at each of times T21 to T26 are illustrated.

[0213] At time T21, when Fig.12 When "ABC" is erased in the state at time T6 in the above-mentioned record control process, the process F2 is executed. Therefore, the record candidate image in which "ABCDE", "FGHIJ" and "KLMNO" are written is recorded as the recorded image 171, and the white pixel area corresponding to the undeleted "DE", "FGHIJ", "KLMNO" in the blackboard area image 172 is set as the recorded area, and the mode transitions to the "erase from board" mode.

[0214] At time T22, the written information has not been further erased from the state at time T21, and the processing of flow F6 in the above-mentioned recording control processing is performed. Therefore, the "erase from plate" mode is maintained.

[0215] At time T23, when "DE" is further erased from the state at time T22, the process F4 in the above-mentioned recording control process is executed. Therefore, in the blackboard area image 172, the white pixel area corresponding to the erased "DE" is released from the recorded area, and the "erase from the board" mode is maintained.

[0216] At time T24, when "P" is written in the place where "A" was written before erasing from the state at time T23, the processing of flow F5 in the above-mentioned record control processing is executed. Therefore, the pixels of the area of ​​the record corresponding to "FGHIJ" and "KLMNO" are masked, and the record candidate image 173 is updated with the blackboard image in which only "P" is written, and the mode transitions to the "writing on the board" mode.

[0217] At time T25, when "QR" is added to the right side of "P" from the state at time T24, the processing of flow F1 in the above-mentioned record control processing is executed. Therefore, the record candidate image 173 is updated with a blackboard image in which only "PQR" is written, in which the pixels corresponding to "FGHIJ" and "KLMNO" in the recorded area are masked, and the "writing on the board" mode is maintained.

[0218] Thereafter, at time T26, when "FGHIJ" is erased from the state at time T25, the process of flow F2 in the above-mentioned recording control process is executed. Therefore, the recording candidate image in which only "PQR" is written is recorded as the recorded image 174, and the mode transitions to the "erase from plate" mode.

[0219] Here, when compared with Fig.19When the mode transition is recorded between recorded image 171 and recorded image 174, "FGHIJ" and "KLMNO" which were once recorded as recorded image 171 are not recorded in recorded image 174 as indicated by the dotted frame. Therefore, a recording candidate image that does not include the writing area in the recorded image is recorded as a recorded image.

[0220] That is, according to the above processing, the recording candidate image including the writing information included in the recorded image that has already been recorded is no longer recorded as the recorded image, and this makes it possible to prevent unnecessary recording.

[0221] In the above example, a value of -1 is set as a recorded flag set for pixels forming a recorded area. Alternatively, it is also possible to allow any value to be set as long as the value can be distinguished from the blackboard writing residual time.

[0222] Incidentally, in the above processing, when a certain amount or more of written information is erased in the “writing on the board” mode (specifically, when the writing area on the board is reduced by 5% or more from the maximum value), the recording candidate image at that time is recorded as the recorded image.

[0223] Therefore, in the case where the writer 4 erases a certain amount or more of the written information in order to correct a writing error or the like while writing the written information 3, a blackboard writing image containing a writing error is recorded, and this results in unnecessary recording.

[0224] Therefore, in the following, an example of preventing a blackboard writing image containing a writing error from being recorded as described above will be described.

[0225] (5-2. Control based on the record of the remaining time of the blackboard writing)

[0226] Fig. 20 1 is a diagram illustrating an example of performing recording control according to a blackboard writing remaining time in order to prevent a blackboard writing image containing a writing error from being recorded.

[0227] Fig. 20 A blackboard writing image, a blackboard writing area image, a pixel information array in which a blackboard writing residual time is set, and a recording candidate image at each of times t31 to t34 are illustrated.

[0228] And in Fig. 20 In the example, a blackboard writing area image and a 5×5 pixel information array are illustrated.

[0229] In the blackboard image at time t31, "ABC" and "DEF" are written as writing information, and in the blackboard area image, pixels (2, 2), (2, 3), (2, 4), (4, 2), (4, 3), and (4, 4) are set as white pixels. At this time, the candidate image is updated with the blackboard image in which "ABC" and "DEF" are written.

[0230] At this time, in the pixel information array, the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) are respectively set to 6, 5 and 4 as the blackboard writing residual time, and the values ​​corresponding to the three pixels (4, 2), (4, 3) and (4, 4) are respectively set to 3, 2 and 1 as the blackboard writing residual time. That is, in the blackboard writing image, "A" is written earliest, and "F" is written most recently.

[0231] When "DEF" is erased from the state at time t31 in the blackboard image at time t32, pixels (4, 2), (4, 3) and (4, 4) are set as black pixels in the blackboard area image, and, although the blackboard area is reduced, the recording candidate image in which "ABC" and "DEF" are written is not recorded as a recorded image.

[0232] At this time, in the pixel information array, according to the time elapsed since the writing of "ABC", the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) are set to 9, 8 and 7 respectively as the remaining time of the blackboard writing, and each of the values ​​corresponding to the three pixels (4, 2), (4, 3) and (4, 4) in which "DEF" has been erased is set to 0 as the remaining time of the blackboard writing.

[0233] In the blackboard image at time t33, when "GHI" is written in the place where "DEF" had been written before erasing from the state at time t32, in the blackboard area image, pixels (4, 2), (4, 3) and (4, 4) are set to white pixels again. The candidate image is updated with the blackboard image in which "ABC" and "GHI" are written.

[0234] At this time, in the pixel information array, the values ​​corresponding to the three pixels (2, 2), (2, 3) and (2, 4) are set to 12, 11 and 10 respectively as the blackboard writing residual time, and the values ​​corresponding to the three pixels (4, 2), (4, 3) and (4, 4) are set to 3, 2 and 1 respectively as the blackboard writing residual time.

[0235] When "AB" and "GH" are erased from the state at time t33 in the blackboard image at time t34, pixels (2, 2), (2, 3), (4, 2) and (4, 3) are set as black pixels in the blackboard area image, and a recording candidate image in which "ABC" and "GHI" are written is recorded as a recorded image.

[0236] At this time, in the pixel information array, each of the values ​​corresponding to the four pixels (2, 2), (2, 3), (4, 2) and (4, 3) is set to 0 as the blackboard writing residual time, and the values ​​corresponding to the two pixels (2, 4) and (4, 4) in which the written information is not deleted are set to -1 as the blackboard writing residual time. That is, the recorded flags are set for the two pixels (2, 4) and (4, 4) forming the recorded area.

[0237] Next, we will refer to Fig.21 The flowchart in describes the details of the above-mentioned record control processing that implements record control based on the remaining time of blackboard writing.

[0238] Fig.21 The processing flow in steps S231 to S243 in Fig.18 The processing flow in steps S131 to S142 in (flows F1, F2, and F3) is basically similar, but the difference is that the processing of step S238 is added.

[0239] Specifically, in the “writing on the board” mode in step S237 , if it is determined that the blackboard writing area has been reduced by 5% or more from the maximum value, the process proceeds to step S238 .

[0240] In step S238, the record control unit 54 determines whether the remaining time of the writing on the blackboard area from which the content has been erased is longer than a predetermined time. Here, for example, it is determined whether the remaining time of the writing on the blackboard area from which the content has been erased is longer than six unit times.

[0241] If it is determined that the writing residual time of the writing area from which the content has been erased is longer than the predetermined time, the process proceeds to step S239, and then the process of flow F2 is executed. That is, the recording candidate image at that time is recorded as the recorded image, and the mode transitions to the "erase from the board" mode.

[0242] On the other hand, in step S238, if it is determined that the remaining time of the blackboard writing in the blackboard writing area from which the content has been erased is not longer (shorter) than the predetermined time, that is, if the remaining time of the blackboard writing in the blackboard writing area from which the content has been erased is equal to or less than six unit times, as Fig. 20 If the time t32 in the image is reached, the process proceeds to step S242, and then the process of flow F3 is performed. That is, the recording candidate image at that time is not recorded as a recorded image.

[0243] Fig. 22 It is a graphic reference Fig.21 The flowchart described in the figure is an example of the mode transition of the blackboard writing state mode in the recording control process, and is a diagram illustrating Fig.12 FIG. 4 is a diagram of an example of a mode transition after time T6 in FIG.

[0244] Fig. 22 Changes in the writing information in the blackboard writing image at each of times T31 and 32 are illustrated.

[0245] At time T31, when Fig.12 When the state at time T6 in the erase mode erases "MNO" in which the blackboard writing residual time is shorter than the predetermined time, the process F3 in the above-mentioned record control process is executed. Therefore, the current value of the blackboard writing area decreases (maintains the maximum value) and maintains the "writing on the board" mode.

[0246] At time T32, in the case where the blackboard writing area after the predetermined time has passed (i.e., "KL" where the blackboard writing remains longer than the predetermined time) is erased from the state at time T31, the processing of flow F2 in the above-mentioned recording control processing is performed. Therefore, the recording candidate image in which "ABCDE", "FGHIJ" and "KLMNO" are written is recorded as the recorded image 211, and the mode transitions to the "erase from the board" mode.

[0247] On the other hand, at time T32, in the case where "STU" is added below the undeleted "KL" from the state at time T31, the processing of flow F1 in the above-mentioned record control processing is performed. Therefore, the record candidate image 221 is updated with the blackboard image in which "ABCDE", "FGHIJ", "KL", and "STU" are written, and the "writing on the board" mode is maintained.

[0248] At this time, the image in which "MNO" is erased before the predetermined time elapses may be recorded as the memo image 222, or may be discarded without being recorded.

[0249] In this way, in Fig. 22 In the mode transition, even if a certain amount or more of written information is erased in the "writing on the board" mode, the candidate image is not updated including the blackboard image of "MNO" that has been erased before the predetermined time has passed.

[0250] That is, according to the above processing, even if the writer 4 erases a certain amount or more of the written information while writing the written information 3 to correct a writing error, etc., a blackboard image containing a writing error is not recorded, and this makes it possible to prevent unnecessary recording.

[0251] <6. Example of computer configuration>

[0252] The above series of processing can be executed not only by hardware but also by software. In the case where the series of processing is executed by software, the program constituting the software is installed from a program recording medium into a computer built into dedicated hardware, a general-purpose personal computer, or the like.

[0253] Fig.23 : is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processes according to a program.

[0254] The information processing device 10 is composed of Fig.23 The configuration shown in is implemented by a computer.

[0255] A central processing unit (CPU) 1001 , a read only memory (ROM) 1002 , and a random access memory (RAM) 1003 are connected to one another via a bus 1004 .

[0256] The bus 1004 is also connected to an input / output interface 1005. The input / output interface 1005 is connected to an input unit 1006 composed of a keyboard, a mouse, etc. and an output unit 1007 composed of a display, a speaker, etc. In addition, the input / output interface 1005 is connected to a storage unit 1008 composed of a hard disk, a nonvolatile memory, etc., a communication unit 1009 composed of a network interface, etc., and a drive 1010 that drives a removable medium 1011.

[0257] To execute the above-described series of processes, the computer configured as described above causes the CPU 1001 to load a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, for example, and then execute the program.

[0258] The program to be executed by the CPU 1001 is recorded on the removable medium 1011 , or is provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed on the storage unit 1008 .

[0259] Note that the program to be executed by the computer may be a program that executes a plurality of processes in the time sequence described in this specification, or may be a program that executes a plurality of processes in parallel or when necessary (for example, when a process is called).

[0260] The embodiments of the present disclosure are not limited to the above-described embodiments and may be modified in various ways within the scope of the present disclosure.

[0261] Furthermore, the effects described herein are merely illustrative and not restrictive, and other effects may be obtained.

[0262] Furthermore, the present disclosure may have the following configurations. (1)

[0264] An information processing device, comprising:

[0265] an area setting unit that sets a writing area including writing information in a writing image showing a writing object on which writing information is written; and

[0266] A recording control unit controls recording of a writing image in response to a change in the writing area. (2)

[0268] The information processing device according to (1), wherein

[0269] The writing area indicates a ratio of a range in which writing information is written to a writing object. (3)

[0271] The information processing device according to (1) or (2), wherein

[0272] The area setting unit sets a writing area by performing expansion processing on writing information in a writing image. (4)

[0274] The information processing device according to (3), wherein

[0275] The area setting unit also performs a resolution reduction process on the writing image in which the writing area has been set. (5)

[0277] The information processing device according to (1) or (2), wherein

[0278] The area setting unit sets a character area, on which character recognition has been performed in a writing image, as a writing area. (6)

[0280] The information processing device according to any one of (1) to (5), wherein

[0281] Recording control unit:

[0282] updating a recording candidate image as a candidate for a recording image using the writing image in response to an increase in the writing area; and

[0283] The recording candidate image is recorded as the recording image in response to the reduction of the writing area. (7)

[0285] The information processing device according to (6), wherein

[0286] The record control unit records, as the record image, a record candidate image excluding a writing area included in the already recorded record image. (8)

[0288] The information processing device according to (7), wherein

[0289] When the writing area has increased from a state in which a part of the writing area remains after the recording image has been recorded, the recording control unit updates the recording candidate image with the writing image shielding the remaining writing area. (9)

[0291] The information processing device according to (8), further comprising:

[0292] A setting unit, when the recorded image has been recorded, sets a recorded flag for pixels in the writing area that has not been deleted in the writing image,

[0293] When the writing area has increased, the recording control unit updates the recording candidate image using the writing image in which pixels to which the recorded flag is set are masked. (10)

[0295] The information processing device according to (6), wherein

[0296] When the writing area has been reduced, the recording control unit determines whether to record the recording candidate image as the recording image based on a remaining time of the writing area that has been erased. (11)

[0298] The information processing device according to (10), further comprising:

[0299] A setting unit is used to set the residual time of the writing area for each pixel of the writing image.

[0300] wherein the recording control unit determines whether to record the recording candidate image as the recording image based on a residual time set for pixels of the erased writing area. (12)

[0302] The information processing device according to (11), wherein

[0303] The recording control unit does not record the recording candidate image as the recording image in a case where the remaining time set for the pixel of the erased writing area is shorter than a predetermined time. (13)

[0305] The information processing device according to (12), wherein

[0306] The recording control unit records the writing area whose remaining time is shorter than a predetermined time as a memo image different from the recorded image. (14)

[0308] The information processing device according to any one of (1) to (13), further comprising:

[0309] An image extraction unit extracts a writing image from a captured image obtained by imaging a writing object. (15)

[0311] The information processing device according to (14), wherein

[0312] A captured image is one frame of a moving image. (16)

[0314] The information processing device according to (14), wherein

[0315] The captured image is a still image. (17)

[0317] An information processing method performed by an information processing device, the method comprising:

[0318] setting a writing area including writing information in a writing image including writing information written on a writing object; and

[0319] The recording of the writing image is controlled in response to the change of the writing area. (18)

[0321] A program for causing a computer to execute a process, the process comprising:

[0322] setting a writing area including writing information in a writing image including writing information written on a writing object; and

[0323] The recording of the writing image is controlled in response to the change of the writing area.

[0324] Reference numerals list

[0325] 1 Blackboard Recording System

[0326] 10 Information processing device

[0327] 20 Input Devices

[0328] 30 Output Devices

[0329] 51 Image acquisition unit

[0330] 52 Blackboard image extraction unit

[0331] 53 Blackboard area setting unit

[0332] 54 Recording control unit

[0333] 55 Storage Units

[0334] 56 Output control unit

[0335] 61 Mode setting unit

[0336] 161 Pixel information setting unit

Claims

1. An information processing device, include: an area setting unit that sets a writing area including the writing information in a writing image showing a writing object on which the writing information is written; as well as a recording control unit that controls recording of the writing image in response to a change in the writing area, Wherein, the recording control unit: updating a record candidate image as a candidate for a record image using the written image in response to an increase in the written area; as well as recording the recording candidate image as the recording image in response to a reduction in the writing area, When the writing area has been reduced, the record control unit determines whether to record the record candidate image as the record image based on a remaining time of the writing area that has been erased.

2. The information processing device according to claim 1, wherein The writing area indicates a ratio of a range in which the writing information is written to the writing object.

3. The information processing device according to claim 2, wherein The area setting unit sets the writing area by performing expansion processing on the writing information in the writing image.

4. The information processing device according to claim 3, wherein The area setting unit also performs a resolution reducing process on the writing image in which the writing area has been set.

5. The information processing device according to claim 1, wherein The area setting unit sets a character area, on which character recognition has been performed in the writing image, as the writing area.

6. The information processing device according to claim 1, wherein The record control unit records the record candidate image excluding the writing area included in the recorded image that has already been recorded as the recorded image.

7. The information processing device according to claim 6, wherein When the writing area has increased from a state in which a part of the writing area remains after the recording image has been recorded, the recording control unit updates the recording candidate image using the writing image shielding the remaining writing area.

8. The information processing device according to claim 7, further comprising: include: a setting unit, when the recorded image has been recorded, the setting unit setting a recorded flag for pixels of the writing area that has not been deleted in the writing image, When the writing area has increased, the recording control unit updates the recording candidate image using the writing image in which pixels provided with the recorded flag are masked.

9. The information processing device according to claim 1, further comprising: include: a setting unit, the setting unit setting a residual time of the writing area for each pixel of the writing image, wherein the record control unit determines whether to record the record candidate image as the record image based on the residual time set for the pixel of the written area that has been erased.

10. The information processing device according to claim 9, wherein The record control unit does not record the record candidate image as the record image in a case where the remaining time set for the pixel of the written area that has been erased is shorter than a predetermined time.

11. The information processing device according to claim 10, wherein The record control unit records the writing area in which the remaining time is shorter than the predetermined time as a memo image different from the record image.

12. The information processing device according to claim 1, further comprising: include: An image extraction unit extracts the writing image from a captured image obtained by imaging the writing object.

13. The information processing device according to claim 12, wherein The captured image is one frame of a moving image.

14. The information processing device according to claim 13, wherein The captured image is a still image.

15. An information processing method executed by an information processing device, the method include: an area setting step of setting a writing area including the writing information in a writing image including the writing information written on the writing object; as well as a recording control step of controlling recording of the writing image in response to a change in the writing area, Wherein, in the recording control step, updating a record candidate image as a candidate for a record image using the written image in response to an increase in the written area; as well as recording the recording candidate image as the recording image in response to a reduction in the writing area, When the writing area has been reduced, whether to record the recording candidate image as the recording image is determined based on a remaining time of the writing area that has been erased.

16. A storage medium storing a program for causing a computer to execute a process, the process include: an area setting step of setting a writing area including the writing information in a writing image including the writing information written on the writing object; as well as a recording control step of controlling the recording of the writing image in response to the change of the writing area, Wherein, in the recording control step, updating a record candidate image as a candidate for a record image using the written image in response to an increase in the written area; as well as recording the recording candidate image as the recording image in response to a reduction in the writing area, When the writing area has been reduced, whether to record the recording candidate image as the recording image is determined based on a remaining time of the writing area that has been erased.