Image forming apparatus

Abstract

To provide an image forming apparatus which does not need to create a record of a conference or the like by itself by forming a voice recognition result on a recording medium.SOLUTION: The image forming device 10 includes a voice-data acquiring unit 33a, a voice recognizing unit 33b, and an image forming unit 220. The sound-data acquiring unit 33a acquires the sound signal Ds. The voice recognizer 33b recognizes the voice Ds and converts it into text data Dc. The image forming unit 220 forms an image representing the character data Dc on the recording medium P.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image forming apparatus. [Background technology]

[0002] Patent Document 1 discloses an integrated media image information storage and retrieval system that processes information supplied from different types of media. This system integrates voice recognition and is also integrated with video systems. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 5,729,741 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the system described in US Pat. No. 6,213,669 does not focus on using speech recognition for transcription and printing.

[0005] An object of the present invention is to provide an image forming apparatus that records the results of speech recognition on a recording medium, thereby eliminating the need to create a record of a meeting or the like by the user. [Means for solving the problem]

[0006] The image forming apparatus of the present invention comprises a voice data acquisition unit, a voice recognition unit, and an image forming unit. The voice data acquisition unit acquires voice data. The voice recognition unit recognizes the voice data and converts it into character data. The image forming unit forms an image representing the character data on a recording medium. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an image forming apparatus that does not need to create a record of a meeting or the like by itself, since the voice recognition result is recorded on a recording medium. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing an image forming apparatus 10 according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the image forming apparatus 10. [Figure 3] FIG. 10 is an explanatory diagram showing an example of a display on the main screen of the ATP application 33. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a display during recording by the ATP application 33. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a display after recording by the ATP application 33. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of minutes printed by the ATP application 33. [Figure 7] 7 is an explanatory diagram illustrating an example of minutes printed in a format different from that shown in FIG. 6. [Figure 8] 4 is a flowchart showing main processing by the image forming apparatus 10. [Figure 9] FIG. 10 is an explanatory diagram illustrating an example in which minutes are created from an audio file. [Figure 10] 10 is an explanatory diagram illustrating a case where minutes are created by recording with a microphone 26. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] Embodiments of the present disclosure will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0010] 1. Configuration of Image Forming Apparatus 10 First, the configuration of an image forming apparatus 10 having an Audio Transcription Program (ATP) function, which is a feature of the present invention, will be described with reference to Figures 1 and 2. Figure 1 is a diagram showing the image forming apparatus 10 according to one embodiment of the present invention. Figure 2 is a block diagram showing the functional configuration of the image forming apparatus 10.

[0011] A "voice transcription program" is a function that recognizes input voice data, automatically transcribes it, and stores the content on a recording medium. For example, it can automatically create minutes of a meeting or other event and print them on paper in a predetermined desired format. Hereinafter, this "voice transcription program" will be referred to as ATP.

[0012] The image forming apparatus 10 is, for example, a copier, a printer, or a multifunction peripheral. In the following, as an example, a case will be described in which the image forming apparatus 10 is a multifunction peripheral having a printer function, a copy function, a facsimile function, and a network communication function.

[0013] 1 and 2, the image forming apparatus 10 includes an image reading unit 11, an image forming unit 12, an operation display unit 23, a FAX communication unit 24, a network communication unit 25, a microphone 26, a storage unit 28, and a control unit 31. Hereinafter, the network communication unit 25 may be referred to as the NW communication unit 25.

[0014] The image reading unit 11 reads an image from an original document M. The image reading unit 11 has an original document table 120 and an original document transport unit 110. The image reading unit 11 reads an image formed on the original document M and generates read data. Specifically, the image reading unit 11 reads an image formed on the original document M transported by the original document transport unit 110, or an image formed on the original document M placed on the original document table 120. Specifically, the image reading unit 11 is composed of an automatic original document feeder, an original document image scanning device (scanner), etc.

[0015] The image forming unit 12 includes an image forming section 220 , a paper feed cassette 230 , a conveying section 240 , and a discharge section 270 .

[0016] The image forming unit 220 forms an image on the recording medium P. For example, the image forming unit 220 forms an image on the recording medium P based on the read data. The image forming unit 220 includes a plurality of toner containers.

[0017] The plurality of toner containers are detachably attached to the image forming apparatus 10. Each of the plurality of toner containers contains toner of a different color. The toner in the toner container is supplied to the image forming unit 220.

[0018] The image forming unit 220 includes an exposure unit, a photosensitive drum, a charging unit, a developing unit, a primary transfer roller, a cleaning unit, an intermediate transfer belt, a secondary transfer roller, and a fixing unit.

[0019] A recording medium P for printing is accommodated in paper feed cassette 230. The recording medium P is transported by transport unit 240. When printing is performed, the recording medium P in paper feed cassette 230 passes through image forming unit 220 and is discharged from discharge unit 270.

[0020] The conveying section 240 conveys the fed recording medium P to the image forming section 220. After the image forming section 220 forms an image on the recording medium P, the conveying section 240 further conveys the recording medium P from the image forming section 220 and discharges the recording medium P to the outside of the image forming apparatus 10.

[0021] The operation display unit 23 is used to allow a user to operate the image forming apparatus 10. The operation display unit 23 includes a display unit 21 and an operation unit 22.

[0022] The display unit 21 has a display screen 21a that displays images. Here, the images include text such as letters and symbols. The display unit 21 is configured with a display such as an LCD (Liquid Crystal Display) or ELD (Electro Luminescence Display) that has a touch panel function.

[0023] The operation unit 22 accepts operation input. In this embodiment, the display unit 21 and a part of the operation unit 22 may be integrated. The operation unit 22 of the operation display unit 23 may also have a touch panel and physical buttons. In this case, the operation unit 22 accepts touch operations on the display screen 21a.

[0024] The FAX communication unit 24 transmits or receives FAX images. Specifically, the FAX communication unit 24 transmits and receives image data and the like to and from other image forming devices and facsimile devices (neither of which are shown) via a network. The received FAX images are printed on a recording medium P by the image forming unit 12. The FAX images are also written as read data into a storage area of ​​the storage unit 28.

[0025] The NW communication unit 25 is capable of communicating with electronic devices equipped with communication devices that use the same communication method (protocol). Specifically, the NW communication unit 25 communicates with other electronic devices via a network such as a LAN (Local Area Network). The NW communication unit 25 is, for example, a communication interface equipped with a communication module such as a LAN board.

[0026] The microphone 26 receives input of voice S and outputs voice data Ds. The microphone 26 receives voice input, for example, by collecting sounds generated around the image forming apparatus 10. The microphone 26 is disposed, for example, in the same area as the operation display unit 23. The microphone 26 outputs a signal indicating the input voice.

[0027] The storage unit 28 is, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 28 may include a random access memory (RAM) and a read only memory (ROM). The storage unit 28 stores various data and a control program for controlling the operation of each unit of the image forming apparatus 10. The control program is executed by the control unit 31. In addition, the image read by the image reading unit 11 is written as read data to a predetermined data area of ​​the storage unit 28.

[0028] The control unit 31 is a hardware circuit configured by a processor such as a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), and the like.

[0029] An OS (Operating System) 32 stored in the storage unit 28 runs on the control unit 31. Examples of the OS 32 include, but are not limited to, Microsoft Windows (registered trademark) and Android (registered trademark). In this embodiment, the ATP function is embodied as an ATP application 33 (hereinafter referred to as "ATP app 33") executed on the OS 32. However, the ATP function is not limited to this implementation method. The ATP app 33 is also stored in the storage unit 28.

[0030] The ATP application 33 includes a voice data acquisition unit 33a, a voice recognition unit 33b, a voice data processing unit 33c, and an image formation control unit 33d.

[0031] The voice data acquisition unit 33a acquires the voice data Ds.

[0032] The voice recognition unit 33b recognizes the voice data Ds and converts it into character data Dc.

[0033] As described above, the image forming unit 220 forms an image on the recording medium P. Therefore, the image forming unit 220 forms an image representing the character data Dc on the recording medium P.

[0034] Therefore, an image representing the character data Dc converted by voice recognition is formed on the recording medium P. As a result, it becomes unnecessary to personally create a record of a meeting or the like.

[0035] As described above, the voice data acquisition unit 33a acquires the voice data Ds. For example, the voice data acquisition unit 33a may acquire the voice data Ds from the microphone 26. Therefore, the voice data acquisition unit 33a acquires the voice S of the moment as the voice data Ds via the microphone 26. As a result, the content of an ongoing conference or the like can be recorded on the recording medium P without having to prepare a voice data file in advance.

[0036] 2. Supplementary information about the configuration of the image forming device 10 and the ATP application 33 Next, a supplementary explanation will be given of the configuration of the image forming apparatus 10 and the ATP application 33 with reference to FIGS. 3 to 7 in addition to FIGS. 1 and 2. FIG. 3 is an explanatory diagram showing an example of a display on the main screen of the ATP application 33. FIG. 4 is an explanatory diagram showing an example of a display during recording by the ATP application 33. FIG. 5 is an explanatory diagram showing an example of a display after recording by the ATP application 33. FIG. 6 is an explanatory diagram showing an example of minutes printed by the ATP application 33. FIG. 7 is an explanatory diagram showing an example of minutes printed in a format different from that of FIG. 6.

[0037] As shown in Fig. 3, the main screen of the ATP application 33 is displayed on the display screen 21a. In this display example, by clicking the microphone-shaped icon displayed on the right side, recording with the microphone 26 can be started. In addition, by performing a touch operation on the display screen 21a, it is possible to switch from recording with the microphone 26 to transcribing an audio file. However, the display content is not limited to this.

[0038] 4, while recording is being performed by the microphone 26, the message "Recording..." indicating that recording is in progress and the contents of the transcription of the audio data Ds at that time are displayed on the screen of the ATP application 33. However, the contents are not limited to this.

[0039] 5, when recording by the microphone 26 ends, the message "Recording Ended" indicating that recording has ended is displayed on the screen of the ATP application 33. Note that, for example, immediately after recording ends, an intermediate progress message may be displayed, such as the fact that the audio data Ds is being converted into character data Dc. However, the display content is not limited to this.

[0040] When printing the speech recognition results on paper or the like, it may be possible to select from a plurality of formats prepared in advance by, for example, operating the operation unit 22. For example, as shown in Fig. 6, character data Dc arranged in chronological order may be printed together with the corresponding time. Alternatively, as shown in Fig. 7, only character data Dc arranged in chronological order may be printed.

[0041] The voice data processor 33c removes noise components from the voice data Ds and extracts portions that match a predetermined specific pattern. Therefore, only the portions that match the predetermined specific pattern are converted into character data Dc. As a result, voice S that does not match the specific pattern is not converted into character data Dc, making the content easier to read.

[0042] For example, when the speech recognition results of a discussion such as a meeting are recorded as minutes on a recording medium, noise components include, but are not limited to, ambient noise, computer operation sounds and sound effects when explaining materials, laughter and applause unrelated to the participants' remarks, abnormal sounds caused by abnormal electrical signals in the microphone or amplifier circuit used during recording, etc. Specific patterns are characteristic patterns of speech, such as, but are not limited to, typical conversation patterns and speech patterns in meetings.

[0043] The image formation control unit 33d causes the image forming unit 220 to form an image representing the character data Dc on the recording medium P based on the operation input received by the operation unit 22. Therefore, the image representing the character data Dc converted by voice recognition is formed on the recording medium P based on the operation input from the user. As a result, the contents of a meeting or the like can be formed on the recording medium P according to the user's convenience or need.

[0044] Examples of operation input include, but are not limited to, touching the display screen 21a. Specifically, examples include, but are not limited to, clicking or tapping on a button (for example, the second button from the right in FIG. 5) arranged on the screen of the ATP application 33. Examples of recording medium P include, but are not limited to, copy paper or printer paper if the image forming device is a multifunction device or printer.

[0045] Alternatively, after the voice data Ds is acquired, the image formation control unit 33d automatically causes the image forming unit 220 to form on the recording medium P an image representing the character data Dc obtained by converting the voice data Ds using the voice recognition unit 33b. Therefore, the image representing the character data Dc converted by voice recognition is automatically formed on the recording medium P after the voice data Ds is acquired. As a result, the contents of a conference or the like can be formed on the recording medium P without requiring any operation by the user.

[0046] As described above, the voice recognition unit 33b recognizes the voice data Ds and converts it into character data Dc. The voice recognition unit 33b may recognize the voice data Ds based on a preset language model and convert it into character data Dc corresponding to the language model. Therefore, by setting a language model in advance, the recognition accuracy of the voice data Ds is improved. As a result, the content recorded on the recording medium P becomes more accurate.

[0047] Examples of languages ​​include, but are not limited to, English and Japanese. A language model is a model that statistically processes a large amount of text in that language, and calculates and predicts, for example, the relationship between a certain word and which word is likely to appear next. Since highly likely word combinations are considered to be accurate sentences, the accuracy of speech recognition is improved. Also, for example, multiple language models may be prepared in advance and stored in the storage unit 28, allowing the language to be recognized to be switched. The switching may be performed automatically or by touching a button or the like arranged on the screen of the ATP application 33.

[0048] 3. Main Processing of Image Forming Apparatus 10 Next, the main processing of the image forming apparatus 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the main processing of the image forming apparatus 10.

[0049] As shown in FIG. 8, this flowchart begins with the voice data acquisition unit 33a acquiring voice data Ds from the OS 32. The voice data Ds may be an existing voice data file or may be generated by the OS 32 itself through live recording. The OS 32 sends the voice data Ds to the ATP application 33, which removes noise and extracts only the necessary parts, such as conversation. The OS 32 then stores the voice data Ds and translates them if necessary. Using a large-scale language model and other functions, the OS 32 transcribes the voice data Ds, allowing the user U to print, save, or store the results in a document box (hereinafter simply referred to as a "box").

[0050] In step S1, the ATP application 33 requests the OS 32 for audio data Ds to be used for transcription. Specifically, the audio data acquisition unit 33a requests the audio data Ds. The audio data Ds is not limited to one, and may be multiple. As described above, the audio data Ds is acquired from a removable device or the like external to the image forming apparatus 10 (step S1a), or from an audio recording process using the microphone 26 to which the audio S is input (step S1b).

[0051] In step S2, the ATP application 33 extracts speech from the audio data Ds. Using several algorithms, the ATP application 33 removes noise components, selects audio based on a certain threshold, and analyzes speech patterns to appropriately extract the audio. Specifically, the audio data processing unit 33c performs processes such as extracting segments from the acquired audio data Ds to be used for speech recognition and transcription. To this end, the audio data processing unit 33c has functions such as pattern detection (step S2a), noise reduction (step S2b), automatic speech recognition (step S2c), audio analysis (step S2d), sensitivity limiter (S2e), and audio identifier (step S2f). However, these functions are not limited to these. Details of each function of the audio data processing unit 33c are set in the audio extraction setting (step S2x).

[0052] In step S3, the control unit 31 stores and rearranges the audio data Ds extracted in step S2.

[0053] In step S4, the ATP application 33 transcribes the extracted portion of the audio data Ds. Using the ATP application 33's internal language model and supported languages, the ATP application 33 analyzes the audio data Ds and converts them into messages to be placed in a document. Specifically, the speech recognition unit 33b recognizes each audio data Ds and converts it into character data Dc (text). To this end, the speech recognition unit 33b has functions such as speaker diarization (step S4a), large-scale language model (step S4b), common / custom vocabulary (step S4c), language recognition (step S4d), profanity filtering (step S4e), and automatic sentence structuring (step S4f). However, these functions are not limited to these. Details of each function of the speech recognition unit 33b are set by language support and settings (step S4x).

[0054] Speaker diarization is the process of obtaining information about who spoke and when (from XX:XX:XX to XX:XX:XX) when there are multiple speakers. Profanity filtering is the process of detecting and filtering inappropriate language. Also known as a NG word filter in English, it removes profane or abusive language.

[0055] In step S5, the control unit 31 stores and rearranges the character data Dc after the text conversion in step S4.

[0056] 4. Example of use of image forming apparatus 10 Next, an example of how the image forming device 10 is used will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is an explanatory diagram illustrating an example of how minutes are created from an audio file. Fig. 10 is an explanatory diagram illustrating an example of how minutes are created by recording with the microphone 26. However, the use is not limited to this example.

[0057] 4.1 When created from an audio file As shown in FIG. 9, the user U first instructs the ATP application 33 to select or open a box or removable media (step S11).

[0058] Upon receiving this instruction, the ATP application 33 requests the OS 32 to access the box or the external device (step S12).

[0059] In response to this request, the OS 32 returns a requested value to the ATP application 33 (step S13). The requested value may be a list of items present in the box or a status requested by the ATP application 33.

[0060] Next, the user U first instructs the ATP application 33 to select a desired file (step S14).

[0061] In response to this, the OS 32 returns the file selected by the user U to the ATP application 33 (step S15).

[0062] The ATP application 33 processes the file returned from the OS 32 and returns a status to the user U (step S16).

[0063] 4.2 When recorded using microphone 26 As shown in FIG. 10, the user U first instructs the ATP application 33 to start recording (step S21).

[0064] Upon receiving this instruction, the ATP application 33 requests the OS 32 to use the microphone 26 (step S22).

[0065] In response to this request, the OS 32 returns the requested value to the ATP application 33 (step S23).

[0066] The ATP application 33 returns the status to the user U (step S24).

[0067] The OS 32 allows the ATP application 33 to use the microphone 26 (step S25).

[0068] This enables the microphone 26 to be used, allowing the user U to record audio (step S26).

[0069] The user U responds with a voice to the ATP application 33 and records it (step S27).

[0070] The microphone 26 returns the microphone data (step S28), which is converted into an audio file that can be read by the OS 32 and passed to the OS 32 (step S29).

[0071] The OS 32 returns the audio file to the ATP application 33 (step S30). Upon receiving this audio file, the ATP application 33 transcribes it.

[0072] The present invention can be embodied in various other forms without departing from its spirit or main features. Therefore, the above-described embodiments are merely illustrative in all respects and should not be interpreted as limiting. The scope of the present invention is defined by the claims and is not limited to the text of the specification. Furthermore, all modifications and variations within the equivalent range of the claims are within the scope of the present invention. [Industrial Applicability]

[0073] The contents of the present invention can be used in an image forming apparatus. [Explanation of symbols]

[0074] 10 Image forming device 12 Image forming unit 21 Display section 22 Control section 23 Operation display section 26 microphones 28 Memory section 31 Control Unit 32 OS 33 ATP App 33a Audio data acquisition unit 33b Voice recognition unit 33c Audio data processing section 33d Image formation control unit 220 Image forming unit

Claims

1. a voice data acquisition unit that acquires voice data; a voice recognition unit that recognizes the voice data and converts it into character data; an image forming unit that forms an image representing the character data on a recording medium; An image forming apparatus comprising:

2. further comprising a microphone that receives voice input and outputs the voice data; The image forming apparatus according to claim 1 , wherein the voice data acquisition unit acquires the voice data from the microphone.

3. an operation unit that accepts operation input; an image forming control unit that causes the image forming unit to form the image representing the character data on the recording medium based on the operation input received by the operation unit; The image forming apparatus according to claim 1 or 2, further comprising:

4. 3. The image forming apparatus according to claim 1, further comprising an image forming control unit that automatically causes the image forming unit to form on the recording medium the image representing the character data converted from the voice data by the voice recognition unit after the voice data has been acquired.

5. 3. The image forming apparatus according to claim 1, further comprising an audio data processing unit that performs processing to remove noise components from the audio data and extract portions that match a predetermined specific pattern.

6. 3. The image forming apparatus according to claim 1, wherein the speech recognition unit recognizes the speech data based on a preset language model and converts the speech data into the character data corresponding to the language model.

Images