Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program, which can reduce time and effort for input work of a user.SOLUTION: An information processing system 1 includes an acquisition section 13, a voice recognition section 111, and a small chalkboard selection section 112. The acquisition section 13 acquires voice data on a voice uttered by a user. The voice recognition section 111 performs voice recognition processing on the voice data. The small chalkboard selection section 112 selects a target form corresponding to a target work from a plurality of small chalkboard forms respectively corresponding to a plurality of types of works. Each of the plurality of small chalkboard forms includes a plurality of input items corresponding to the types of works. The voice recognition section 111 performs voice recognition on the voice data to recognize at least a part of a plurality of pieces of input information respectively input to the plurality of input items, and the plurality of input items. The small chalkboard selection section 112 selects the target form from the plurality of small chalkboard forms on the basis of a recognition result of the voice recognition section 111.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. More specifically, the present disclosure relates to an information processing system, an information processing method, and a program that perform information processing related to creating a small blackboard (electronic small blackboard) to be embedded in a photographed image of a construction site. [Background technology]

[0002] Patent Document 1 discloses a photographing terminal that takes construction photos. The display unit of the photographing terminal displays a layout diagram in which the locations where equipment will be installed are indicated by icons. When a user taps an icon on the layout diagram, a launch unit of the photographing terminal launches a camera function and takes a photo of the construction target area. An attachment unit of the photographing terminal attaches the construction information linked to the icon tapped by the user to the image taken with the camera function and sends the image to a server. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-141388 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, with the digitalization of construction photographs, a small blackboard (electronic small blackboard) on which the details of the work related to the construction are written has been added to the construction photograph. When taking construction photographs using the photographing terminal described in Patent Document 1, if a photograph is taken of equipment at an installation location that has not been registered in advance, the user must select a small blackboard of an appropriate format to write the details of the work and perform input work, which has been time-consuming for the user's input work.

[0005] An object of the present disclosure is to provide an information processing system, an information processing method, and a program that can reduce the effort required for user input work. [Means for solving the problem]

[0006] An information processing system according to one aspect of the present disclosure includes an acquisition unit, a voice recognition unit, and a small blackboard selection unit. The acquisition unit acquires voice data of a voice uttered by a user. The voice recognition unit performs voice recognition processing on the voice data. The small blackboard selection unit selects a target format corresponding to a target task from a plurality of small blackboard formats, each corresponding to a plurality of types of task. Each of the plurality of small blackboard formats includes a plurality of input items according to the type of task. The voice recognition unit recognizes the voice data by voice recognition, thereby recognizing at least a portion of the plurality of input items and the plurality of pieces of input information entered into each of the plurality of input items. The small blackboard selection unit selects the target format from the plurality of small blackboard formats based on the recognition result of the voice recognition unit.

[0007] An information processing method of one aspect of the present disclosure includes an acquisition step, a voice recognition step, and a small blackboard selection step. In the acquisition step, voice data of a voice uttered by a user is acquired. In the voice recognition step, voice recognition processing is performed on the voice data. In the small blackboard selection step, a target format corresponding to a target task is selected from a plurality of small blackboard formats corresponding to a plurality of types of tasks, respectively. Each of the plurality of small blackboard formats includes a plurality of input items according to the type of task. In the voice recognition step, the voice data is voice recognized to recognize at least a portion of the plurality of input items and the plurality of pieces of input information entered into each of the plurality of input items. In the small blackboard selection step, the target format is selected from the plurality of small blackboard formats based on the recognition result in the voice recognition step.

[0008] A program according to one aspect of the present disclosure is a program for causing one or more processors of a computer system to execute the information processing method. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to reduce the effort required for the user to perform input processing. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an information processing system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing an example of a screen display of the information processing system. [Figure 3] FIG. 3 is an explanatory diagram illustrating an example of a composite image of the information processing system. [Figure 4] FIG. 4 is an explanatory diagram showing a first example of the small blackboard format of the information processing system. [Figure 5] FIG. 5 is an explanatory diagram showing a second example of the small blackboard format of the information processing system. [Figure 6] FIG. 6 is an explanatory diagram showing a third example of the small blackboard format of the information processing system. [Figure 7] FIG. 7 is an explanatory diagram showing an example of a small blackboard created by the information processing system. [Figure 8] FIG. 8 is a flowchart illustrating an example of processing in the information processing system. DETAILED DESCRIPTION OF THE INVENTION

[0011] (Embodiment) An information processing system 1, an information processing method, and a program according to an embodiment will be described below with reference to the accompanying drawings. However, the embodiment described below is merely one of various embodiments of the present disclosure. The embodiment described below can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved.

[0012] (overview) The "target space" referred to in this disclosure is a construction site of a building (e.g., an apartment building, a detached house, an office building, a hospital, a factory, a commercial facility, etc.) before, during, or after construction. However, the "target space" is not limited to this example.

[0013] In construction work such as new construction, remodeling, or renovation of a building, a part of the target space (construction site) of the building to be constructed is photographed, and the status of the building and the construction status are confirmed and inspected using the photographed images. The information processing system 1 presents information related to such photographing work to the worker (user) to assist in the confirmation and inspection of the status of the building and the construction status. In other words, the information processing system 1 assists in the confirmation and inspection associated with the construction of the building.

[0014] The information processing system 1 has a function to process photographed images of a part of the target space of a building and information related to the photographed images in order to assist in confirmation and inspection associated with the construction of the building. Specifically, the information processing system 1 has a function to create a small blackboard (electronic small blackboard) to be combined with a photographed image of the construction site. Users such as workers and inspectors take photographed images of various parts of the target space before, during, or after construction. The small blackboard is created by inputting work information related to the construction at the construction site into a small blackboard (electronic small blackboard) format (target format) corresponding to the construction site to be photographed.

[0015] In this embodiment, as an example, the building is an apartment building, and a dwelling unit in the apartment building is being renovated. In this case, the target space is the dwelling unit (construction site) of the apartment building, which is the construction site for the renovation. When renovating a dwelling unit, a current situation survey is first conducted to confirm the condition of the dwelling unit before the renovation. Then, demolition work is carried out to remove unnecessary parts and equipment from the current dwelling unit, and the main renovation work is carried out. The main work includes carpentry work, equipment work, piping work, wiring work, electrical work, painting work, interior work, etc. Once the main work is completed, a final inspection is carried out to inspect the dwelling unit after the renovation. During each of the current situation survey, demolition work, main work, and completion inspection, photography may be carried out to confirm and inspect the condition of the building and the construction status, etc.

[0016] As shown in FIG. 1, the information processing system 1 of this embodiment includes an acquisition unit 13, a voice recognition unit 111, and a small blackboard selection unit 112. The acquisition unit 13 acquires voice data of a voice uttered by a user. The voice recognition unit 111 performs voice recognition processing on the voice data. The small blackboard selection unit 112 selects a target form corresponding to a target task from a plurality of small blackboard forms A4 (see FIGS. 4 to 6) corresponding to a plurality of types of tasks, respectively. Each of the plurality of small blackboard forms A4 includes a plurality of input items D11 to D14 corresponding to the type of task. The voice recognition unit 111 recognizes the plurality of input items D11 to D14 and at least a portion of the plurality of pieces of input information entered into the plurality of input items D11 to D14 by voice recognition of the voice data. The small blackboard selection unit 112 selects a target form from the plurality of small blackboard forms A4 based on the recognition result of the voice recognition unit 111. In the following description, when the three types of small blackboard format A4 shown in FIGS. 4 to 6 are individually described, the small blackboard format A4 shown in FIGS. 4 to 6 may be referred to as small blackboard formats A41, A42, and A43, respectively.

[0017] According to this embodiment, the small blackboard selection unit 112 selects the target format based on the recognition result of the speech recognition unit 111 performing speech recognition on at least a portion of the plurality of input items D11 to D14 and the plurality of pieces of input information. Therefore, the small blackboard selection unit 112 can select an appropriate small blackboard format A4 as the target format from the plurality of small blackboard formats A4 based on the recognition result of the speech recognition unit 111, thereby eliminating the need for the user to select the small blackboard format A4 himself and reducing the effort required for the user's input processing.

[0018] Furthermore, functions similar to those of the information processing system 1 can be embodied in an information processing method. The information processing method of this embodiment is an information processing method executed by the information processing system 1, and includes an acquisition step, a voice recognition step, and a small blackboard selection step. In the acquisition step, voice data of a voice uttered by a user is acquired. In the voice recognition step, voice recognition processing is performed on the voice data. In the small blackboard selection step, a target form corresponding to a target task is selected from a plurality of small blackboard forms A4 corresponding to a plurality of types of tasks, respectively. Each of the plurality of small blackboard forms A4 includes a plurality of input items D11 to D14 (see FIGS. 4 to 6) corresponding to the type of task. In the voice recognition step, voice recognition is performed on the voice data to recognize at least a portion of the plurality of input items D11 to D14 and the plurality of pieces of input information entered into the plurality of input items D11 to D14, respectively. In the small blackboard selection step, a target form is selected from the plurality of small blackboard forms A4 based on the recognition result in the voice recognition step.

[0019] The information processing method can be realized by a program. The program of this embodiment is a program for causing one or more processors of a computer system to execute the information processing method. The program may be recorded on a non-transitory recording medium readable by the computer system.

[0020] (detail) (1) Overall structure First, the overall configuration of an information processing system 1 will be described with reference to FIG.

[0021] The information processing system 1 is, for example, a tablet terminal, a smartphone, or a dedicated construction photo terminal used for taking construction photos.

[0022] The information processing system 1 includes a computer system having one or more processors and a memory. At least some of the functions of the information processing system 1 are realized by the processor of the computer system executing a program recorded in the memory of the computer system. The program may be recorded in the memory, or may be provided via a telecommunications line such as the Internet, or may be provided by recording it on a non-transitory recording medium such as a memory card.

[0023] As shown in FIG. 1, the information processing system 1 includes a processing unit 11, a storage unit 12, an acquisition unit 13, an operation unit 14, a display unit 15, an imaging unit 16, and a communication unit 17.

[0024] (2) Processing section The processing unit 11 includes one or more processors of the information processing system 1. The processing unit 11 executes a program to realize a predetermined function.

[0025] The processing unit 11 is configured to perform overall control of the information processing system 1. For example, the processing unit 11 controls the display of images by the display unit 15. The processing unit 11 also controls the photographing operation by the photographing unit 16.

[0026] In this embodiment, the processing unit 11 has functions such as a voice recognition unit 111, a small blackboard selection unit 112, a small blackboard creation unit 113, and an image synthesis unit 114. Note that the voice recognition unit 111, the small blackboard selection unit 112, the small blackboard creation unit 113, and the image synthesis unit 114 merely indicate functions realized by the processing unit 11, and do not indicate actual configurations. Each function of the processing unit 11 will be described below.

[0027] (2.1) Voice Recognition Unit The voice recognition unit 111 performs voice recognition on the voice data acquired by the acquisition unit 13, for example, using a voice recognition dictionary file stored in the storage unit 12. The voice recognition dictionary file stored in the storage unit 12 includes data on words related to construction work being carried out in the target space, as well as words related to input items and input information on an electronic blackboard to be displayed overlaid on a construction photograph. The voice recognition dictionary file stored in the storage unit 12 may further include word data on operation commands for causing the information processing system 1 to perform a desired operation.

[0028] It is not essential that the storage unit 12 of the information processing system 1 stores a voice dictionary file, and the voice recognition unit 111 may perform voice recognition using a voice dictionary file provided on the cloud.

[0029] (2.2) Small Blackboard Selection Section The small blackboard selection unit 112 selects a small blackboard form A4 corresponding to a target task from a plurality of small blackboard forms A4 corresponding to a plurality of types of tasks based on the recognition result of the voice recognition unit 111.

[0030] For example, if the recognition result of the voice recognition unit 111 includes information on one or more input items, the small blackboard selection unit 112 selects the small blackboard form A4 having one or more input items included in the recognition result as the target form, which is the small blackboard form for the target task.

[0031] Furthermore, if the recognition result of the voice recognition unit 111 includes one or more pieces of input information corresponding to one or more input items, the small blackboard selection unit 112 determines one or more corresponding input items from the one or more pieces of input information obtained by voice recognition.Then, the small blackboard selection unit 112 selects the small blackboard form A4 including one or more input items as the target form, which is the small blackboard form for the target task.

[0032] Furthermore, if the recognition result of the voice recognition unit 111 includes one or more first input items and one or more pieces of second input information corresponding to one or more second input items, the small blackboard selection unit 112 determines one or more corresponding second input items from the one or more pieces of second input information obtained by voice recognition.The small blackboard selection unit 112 then selects a small blackboard format A4 including one or more first input items and one or more second input items as the target format, which is the small blackboard format for the target task.

[0033] In addition, if the recognition result of the voice recognition unit 111 alone cannot narrow down the target format to one, the small blackboard selection unit 112 may select two or more candidate formats based on the recognition result of the voice recognition unit 111. In other words, the small blackboard selection unit 112 may perform processing to narrow down multiple small blackboard formats A4 to two or more candidate formats based on the recognition result of the voice recognition unit 111. In this case, the processing unit 11 displays the two or more candidate formats selected by the small blackboard selection unit 112 on the display unit 15 and accepts a selection operation by the user. When the operation unit 14 accepts a selection operation by the user to select a target format from two or more candidate formats, the small blackboard selection unit 112 selects a target format corresponding to the target task based on the selection operation.

[0034] In this way, the small blackboard selection unit 112 selects two or more candidate forms that are candidates for the target form from the multiple small blackboard forms A4 based on the recognition result of the voice recognition unit 111. Then, when the operation unit 14 receives a user's selection operation to select a target form from two or more candidate forms, the small blackboard selection unit 112 selects a target form that corresponds to the target construction project based on the selection operation. Because the candidate forms are narrowed down by the small blackboard selection unit 112, the user's effort in selecting a target form can be reduced compared to when the user selects a target form from multiple small blackboard forms.

[0035] (2.3) Small Blackboard Creation Department The small blackboard creation unit 113 creates the small blackboard A5 by inputting multiple pieces of input information into multiple input fields included in the target form selected by the small blackboard selection unit 112. Specifically, the voice recognition unit 111 recognizes the multiple pieces of input information to be input into the multiple input fields by voice recognizing the voice data, and the small blackboard creation unit 113 creates the small blackboard A5 by inputting the multiple pieces of input information into the multiple input fields based on the recognition results of the voice recognition unit 111. In other words, the small blackboard creation unit 113 creates text data of the input information based on the recognition results of the voice recognition unit 111 and displays this text data in the input fields of the corresponding input fields, thereby creating image data for the small blackboard A5.

[0036] Fig. 7 shows an example of image data of a small blackboard A5 created by the small blackboard creation unit 113. The small blackboard creation unit 113 creates image data of a small blackboard A5 that displays, for example, text data of input information in the second display area AR2 of the small blackboard format A41 shown in Fig. 4, and text data indicating the name of the object to be photographed (e.g., a distribution board) in the third display area AR3.

[0037] (2.4) Image synthesis unit The image synthesis unit 114 synthesizes the captured image A3 captured at the shooting position A2 with the image data of the small blackboard A5 created by the small blackboard creation unit 113, thereby creating a composite image A6 (see FIG. 3) with the small blackboard A5 added. The image synthesis unit 114 stores the created composite image A6 in the storage unit 12.

[0038] The position of the small blackboard A5 added to the captured image A3 can be changed as needed, and for example, it is preferable to place the small blackboard A5 in a position that does not overlap with the subject captured at the capturing position A2. The composite image A6 shown in Fig. 3 is an image obtained by combining the image data of the small blackboard A5 with the captured image A3 of the distribution board, and the image data of the small blackboard A5 is combined in a position that does not overlap with the distribution board.

[0039] (3) Storage section The storage unit 12 stores information used in the information processing system 1. The storage unit 12 stores, for example, a space diagram A1, a photographing position A2, a photographed image A3, and a plurality of small blackboard forms A4.

[0040] The space diagram A1 is a diagram representing the target space. The space diagram A1 is, for example, a floor plan of the target space, and if the target space is a dwelling unit, it may be a floor plan of the dwelling unit. The space diagram A1 may be a two-dimensional drawing, a three-dimensional drawing, or a virtual image representing the target space.

[0041] The photographing position A2 is a position where the photographing unit 16 photographs a photographing target present in the target space, and is set on the space map A1. The photographing position A2 may be set in advance on the space map A1, or the user may add or delete the photographing position A2. In the example screen shown in FIG. 2, multiple photographing positions A2 are set on the space map A1, but the number of photographing positions A2 may be one or more and can be changed as appropriate. In the example screen of FIG. 2, six markers M1 to M6 representing six photographing positions A2 respectively are displayed, but the number of markers can be changed as appropriate depending on the number of photographing positions A2. In the following description, the multiple markers M1 to M4 displayed superimposed on the space map A1 may be collectively referred to as marker Mn (n is a natural number).

[0042] The storage unit 12 stores the captured image A3 captured at each of the multiple capturing positions A2 by the imaging unit 16. The storage unit 12 stores the captured image A3 in association with information about the capturing position A2 at which the captured image A3 was captured.

[0043] The memory unit 12 stores a plurality of small blackboard forms A4 in advance. There are many types of construction work to be carried out in the target space, including carpentry work, equipment work, piping work, wiring work, electrical work, painting work, and interior construction work. Each type of construction work involves many different tasks, such as checking the current status, checking the removal status, checking the delivery of materials, recording the construction status, and recording the post-construction status. Because the input items to be written on the small blackboard A5 to be superimposed on the photographed image A3 of the construction site vary depending on the type of work, the memory unit 12 stores a plurality of small blackboard forms A4 corresponding to a plurality of types of work. Each of the plurality of small blackboard forms A4 has a rectangular display area modeled after a conventional construction blackboard, and is provided with a plurality of input items for entering information about the corresponding work.

[0044] Here, the multiple input items include subject, photo category, type of work, location, etc. The name of the work to be carried out on the target space is input in "subject." Information indicating the timing of photographing the shooting location A2 (before removal, before construction, during construction, after construction, etc.) is input in "photo category." Information indicating the type of work to be carried out at the shooting location A2 (for example, carpentry work, equipment work, piping work, wiring work, electrical work, painting work, interior work, etc.) is input in "type of work." The name of the shooting location A2 is input in "location."

[0045] 4 to 6 show an example of a small blackboard form A4. A first display area AR1 is provided in the upper left of each small blackboard form A4, displaying the names of multiple input items. A second display area AR2 is provided in the upper right of each small blackboard form A4, displaying multiple pieces of input information entered for each of the multiple input items. A third display area AR3 is provided below each small blackboard form A4, displaying the names of items to be worked on, etc.

[0046] In the small blackboard form A41 shown in Fig. 4, four input items D11 to D14, namely, subject, photo category, type of work, and location, are displayed in the first display area AR1. In the small blackboard form A42 shown in Fig. 5, three input items D11 to D13, namely, subject, photo category, and type of work, are displayed in the first display area AR1. In the small blackboard form A43 shown in Fig. 6, two input items D11 and D13, namely, subject and type of work, are displayed in the first display area AR1.

[0047] The storage unit 12 also stores image data of the small blackboard (electronic small blackboard) A5 created by the small blackboard creation unit 113 in association with the photographing position A2. The storage unit 12 also stores a composite image A6 created by combining the photographed image A3 taken by the photographing unit 16 with the image data of the small blackboard A5. The storage unit 12 stores the composite image A6 in association with the photographing position A2 of the photographed image A3.

[0048] The storage unit 12 preferably has a rewritable nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash memory.

[0049] (4) Acquisition department The acquisition unit 13 includes a microphone that converts the voice uttered by the user into electrical voice data, and outputs the voice data to the voice recognition unit 111 .

[0050] When taking a photographed image A3 at the photographing position A2, if the user vocalizes the input items or input information for the small blackboard A5 to be superimposed on the photographed image A3, the acquisition unit 13 converts the voice uttered by the user into voice data and outputs it to the voice recognition unit 111.

[0051] Note that even when the user issues an operation command to operate the information processing system 1 by voice, the acquisition unit 13 converts the operation command into voice data and outputs it to the voice recognition unit 111. When the voice recognition unit 111 recognizes the voice data of the operation command, the processing unit 11 operates according to the content of the operation command, so that the information processing system 1 can be operated by voice.

[0052] It is not essential that the acquisition unit 13 includes a microphone, and the acquisition unit 13 may acquire audio data input from an external microphone.

[0053] (5)Operation section The operation unit 14 is a user interface device that accepts user operations (user operations). For example, the operation unit 14 includes at least one of a touch panel, a button, and a key switch that accepts user operations (manual operations) by hand.

[0054] When the operation unit 14 receives a user operation, it outputs an operation signal corresponding to the user operation to the processing unit 11. The processing unit 11 performs processing corresponding to the operation signal. For example, the processing unit 11 controls the display unit 15 or the imaging unit 16 in response to the operation signal.

[0055] When the operation unit 14 has a touch panel that accepts manual operations, user operations on the touch panel include, for example, tapping, double tapping, long tapping, dragging, swiping, flicking, pinching in, and pinching out.

[0056] (6) Photography Department The photographing unit 16 is equipped with a two-dimensional image sensor such as a CCD (Charge Coupled Devices) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The photographing unit 16 functions as a camera that captures color or monochrome images (still images) and generates data for the photographed image A3. The photographing unit 16 is used to photograph the construction site of the target construction work. The photographing unit 16 stores the data for the photographed image A3 in the memory unit 12.

[0057] For example, if a touch panel display realizes the functions of both the operation unit 14 and the display unit 15, when the user taps a marker Mn displayed on the screen of the display unit 15, the photographing unit 16 enters a state of waiting for photographing. Furthermore, when the user performs a predetermined operation on the operation unit 14, the photographing unit 16 takes a photograph. The storage unit 12 stores data of the photographed image A3 in association with information on the photographing position A2 corresponding to the marker Mn tapped by the user.

[0058] (7) Communications Department The communication unit 17 includes a communication interface device. The communication unit 17 is capable of communicating with an external system via the communication interface device. In this disclosure, "capable of communication" means being able to send and receive signals directly or indirectly via a network or a repeater, etc., using an appropriate communication method such as wired communication or wireless communication. Note that wired communication refers to wired communication via, for example, a twisted pair cable, a dedicated communication line, or a LAN (Local Area Network) cable. Wireless communication refers to wireless communication that complies with standards such as Wi-Fi (registered trademark) or low-power wireless (specified low-power wireless) that does not require a license.

[0059] The communication unit 17 exchanges data with an external system. For example, the communication unit 17 acquires data of a space diagram A1 from an external system. For example, the communication unit 17 acquires data of the space diagram A1, which is a diagram of a target space, from an external CAD system or an external server, and stores the data of the space diagram A1 in the storage unit 12. Note that the space diagram A1 in this embodiment is a floor plan of a dwelling unit, as shown in FIG. 2.

[0060] (8) Display section The display unit 15 is a display device using an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diode), or the like, and has a screen for displaying images. The display unit 15 is controlled by the processing unit 11.

[0061] The touch panel display may function as both the operation unit 14 and the display unit 15.

[0062] As shown in FIG. 2, the display unit 15 displays a space map A1 that is a diagram of the target space. Specifically, the display unit 15 displays the space map A1 on which a plurality of markers Mn (M1 to M6) are superimposed. The plurality of markers Mn correspond one-to-one to a plurality of photographing positions A2. Each of the plurality of markers Mn is displayed at the same position as the corresponding photographing position A2, and represents the location of the corresponding photographing position A2. Note that the display position of the marker Mn is, for example, the same position as the corresponding photographing position A2, but is not limited to a position that exactly matches the photographing position A2, and may be near the photographing position A2.

[0063] Note that sequence information indicating the photographing sequence of the plurality of photographing positions A2 is displayed superimposed on each of the plurality of markers Mn (M1 to M6). In this embodiment, the sequence information of the plurality of photographing positions A2 is represented by consecutive numbers such as 1, 2, 3, ...

[0064] For example, a marker M1 and the number [1] as sequence information are displayed superimposed on the spatial diagram A1 at a position corresponding to the photographing position A2 where the first photograph is taken. Similarly, a marker Mn and the number [n] as sequence information are displayed superimposed on the spatial diagram A1 at a position corresponding to the photographing position A2 where the nth photograph is taken (n is a natural number).

[0065] The display unit 15 is also used to display the image data of the small blackboard A5, and is also used to display the composite image A6.

[0066] (9) Processing flow Next, the flow of processing by the information processing system 1 in the work of taking photos for construction confirmation will be described with reference to Fig. 8. Note that the flowchart shown in Fig. 8 is merely an example, and the order of each step may be changed as appropriate, and steps may be added or omitted as appropriate.

[0067] The information processing system 1 receives input of a space diagram A1, a plurality of photographing positions A2, and a plurality of small blackboard forms A4 (step ST1), and the storage unit 12 stores the information input in step ST1.

[0068] When the necessary information is input in step ST1, the information processing system 1 starts processing to display the space map A1 on the display unit 15. That is, the processing unit 11 performs display processing to display the space map A1 in which markers Mn are superimposed on a plurality of shooting positions A2 on the display unit 15 (step ST2).

[0069] When the display unit 15 is displaying the space diagram A1, and the user taps the marker Mn or taps any position on the space diagram A1 as a new shooting position A2 (step ST3: Yes), the operation unit 14 accepts the operation information, and the processing unit 11 starts the shooting unit 16. After that, when the operation unit 14 accepts a predetermined operation, the shooting unit 16 takes an image of the shooting target (step ST4), and the captured image A3 taken by the shooting unit 16 is stored in the memory unit 12.

[0070] When the photographing unit 16 performs the photographing process, the processing unit 11 starts the process of creating a small blackboard A5 to be combined with the photographed image A3.

[0071] The information processing system 1 waits for voice input of input items to be displayed on the small blackboard A5 or input information related to the input items (step ST5). When the user utters the input items or input information by voice (step ST5: Yes), the acquisition unit 13 acquires the voice data, and the voice recognition unit 111 performs voice recognition processing (step ST6).

[0072] At this time, the small blackboard selection unit 112 performs a selection process to select the small blackboard form A4 containing the input items entered by voice or the input items corresponding to the input information as the target form corresponding to the target task based on the recognition results of the voice recognition unit 111 (step ST7).

[0073] In the selection process of step ST7, if the target format is narrowed down to one (step ST8: Yes), the processing unit 11 causes the display unit 15 to display an image of the target format (step ST12).

[0074] If the selection process of step ST7 fails to narrow down the target formats to one (step ST8: No), the small blackboard selection unit 112 selects two or more small blackboard formats A4 that include input items input by voice or input items corresponding to the input information as candidate formats that are candidates for the target format. At this time, the processing unit 11 displays images of the two or more candidate formats on the display unit 15 (step ST9) and waits for a selection operation by the user (step ST10). When the user performs a selection operation to select a target format from the two or more candidate formats displayed on the display unit 15 using, for example, the operation unit 14 (step ST10: Yes), the operation unit 14 accepts the user's selection operation. Then, the small blackboard selection unit 112 determines the candidate format selected by the user as the target format based on the operation information accepted by the operation unit 14 (step ST11), and the processing unit 11 displays an image of the target format on the display unit 15 (step ST12).

[0075] When the image of the target form is displayed on the display unit 15, the user vocally inputs input information for input items for which no input information has been input, among the multiple input items provided in the target form. The information processing system 1 waits for the vocal input of input information (step ST13), and when the user vocally inputs the input information, the voice recognition unit 111 performs voice recognition processing (step ST14).

[0076] When the input information for all of the input items provided in the target form is input by voice, the small blackboard creation unit 113 creates image data of the small blackboard A5 in which the text data of the input information is displayed in the input fields corresponding to the input items (step ST15). The small blackboard creation unit 113 stores the created image data of the small blackboard A5 in the storage unit 12.

[0077] Then, the image synthesis unit 114 creates a synthetic image A6 by superimposing the image data of the small blackboard A5 on the photographed image A3 photographed by the photographing unit 16 (step ST16), and stores the created synthetic image A6 in the storage unit 12. Then, the processing unit 11 displays the synthetic image A6 on the display unit 15 (step ST17), allowing the user to check the created synthetic image A6.

[0078] Here, when the operation unit 14 accepts an end operation by the user (step ST18: Yes), the processing unit 11 ends the process of displaying the space diagram A1 on the display unit 15.

[0079] On the other hand, when the operation unit 14 accepts a predetermined operation by the user (step ST19: Yes), the processing unit 11 returns to step ST2 and causes the space map A1 to be displayed on the display unit 15. When the display unit 15 is displaying the space map A1, and the user taps the marker Mn or taps any position on the space map A1 as a new shooting position A2 (step ST3: Yes), the information processing system 1 repeatedly executes the processes from step ST4 onwards described above.

[0080] In this way, when the user utters an input item in step ST5, the small blackboard selection unit 112 selects the target format based on the recognition result of the voice recognition unit 111. Thereafter, in step ST13, when the user utters input information related to the input item, the small blackboard creation unit 113 identifies the input information for the input item based on the recognition result of the voice recognition unit 111, and creates a small blackboard A5 on which text data or graphic symbols indicating the input information are displayed in the input field of the target format.

[0081] In step ST5, if the user speaks input information corresponding to an input item, the small blackboard selection unit 112 identifies the input item corresponding to the input information based on the recognition result of the voice recognition unit 111, and selects a target form including the input item. Then, the small blackboard creation unit 113 creates a small blackboard A5 on which text data or graphic symbols indicating the input information spoken by the user in step ST5 are displayed in the input field of the target form. In this case, the user only needs to speak the input information corresponding to the input item, further reducing the user's input work.

[0082] (Modification of the embodiment) Modifications of the embodiment are listed below. The following modifications may be implemented in appropriate combination.

[0083] The shape of the marker Mn is not limited to the shape of a pin, but may be any shape.

[0084] The order information displayed by superimposing it on the markers Mn is not limited to numbers, and may be, for example, letters such as the alphabet, or figures. The order in which the images are taken may be represented by the color of the markers Mn. In other words, the order information may be indicated by the color of the markers Mn.

[0085] Furthermore, the display unit 15 may display the marker Mn representing the shooting position A2 where image shooting has not yet been completed and the marker Mn representing the shooting position A2 where image shooting has been completed in different display modes (for example, colors, shapes, etc.).

[0086] The information processing system 1 or the information processing method according to the present disclosure includes a computer system. The computer system is primarily composed of a processor and memory as hardware. At least a portion of the functions of the information processing system 1 or the information processing method according to the present disclosure are realized by the processor executing a program stored in the memory of the computer system. The program may be pre-stored in the memory of the computer system, provided via a telecommunications line, or provided on a non-transitory recording medium readable by the computer system, such as a memory card, optical disk, or hard disk drive. The processor of the computer system is composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integrations (VLSIs), or ultra-large-scale integrations (ULSIs). Furthermore, field-programmable gate arrays (FPGAs), which are programmable after the LSI is manufactured, or logic devices capable of reconfiguring the connections within the LSI or the circuit partitions within the LSI, can also be used as processors. The electronic circuits may be integrated into one chip or distributed across multiple chips. The chips may be integrated into one device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

[0087] Furthermore, it is not essential for the information processing system 1 that multiple functions are integrated into one device, and multiple components of the information processing system 1 may be distributed across multiple devices. For example, a device including the image capturing unit 16 may be provided separately from a device including the display unit 15.

[0088] At least some of the functions of the information processing system 1, for example, the image processing function in the processing unit 11, may be realized by a server or the cloud (cloud computing), or the like.

[0089] (summary) As described above, the information processing system (1) of the first aspect includes an acquisition unit (13), a voice recognition unit (111), and a small blackboard selection unit (112). The acquisition unit (13) acquires voice data of a voice uttered by a user. The voice recognition unit (111) performs voice recognition processing on the voice data. The small blackboard selection unit (112) selects a target form corresponding to a target task from a plurality of small blackboard forms (A4) corresponding to a plurality of types of tasks, respectively. Each of the plurality of small blackboard forms (A4) includes a plurality of input items (D11 to D14) corresponding to the type of task. The voice recognition unit (111) recognizes the plurality of input items (D11 to D14) and at least a portion of the plurality of pieces of input information entered into the plurality of input items (D11 to D14) by voice recognition of the voice data. The small blackboard selection unit (112) selects a target form from the plurality of small blackboard forms (A4) based on the recognition result of the voice recognition unit (111).

[0090] According to this aspect, it is possible to reduce the effort required for the user to perform input operations.

[0091] The information processing system (1) of the second aspect is the same as that of the first aspect, and further includes an operation unit (14) that accepts user operations. The small blackboard selection unit (112) selects two or more candidate styles that are candidates for a target style from a plurality of small blackboard styles (A4) based on the recognition result of the voice recognition unit (111). When the operation unit (14) accepts a user's selection operation to select a target style from two or more candidate styles, the small blackboard selection unit (112) selects a target style corresponding to the target task based on the selection operation.

[0092] According to this aspect, the user selects the target style from two or more candidate styles narrowed down by the small blackboard selection unit (112), thereby reducing the effort required for the user to perform the selection operation.

[0093] The information processing system (1) of the third aspect, in the first or second aspect, further includes a small blackboard creation unit (113) that creates a small blackboard (A5) by inputting multiple pieces of input information into multiple input items (D11 to D14) included in the target form.

[0094] According to this aspect, it is possible to reduce the effort required for the user to perform input operations.

[0095] In the information processing system (1) of the fourth aspect, in the third aspect, the voice recognition unit (111) recognizes a plurality of pieces of input information input into a plurality of input items (D11 to D14) by voice recognition of voice data. The small blackboard creation unit (113) creates a small blackboard (A5) by inputting the plurality of pieces of input information into the plurality of input items (D11 to D14) based on the recognition result of the voice recognition unit (111).

[0096] According to this aspect, it is possible to reduce the effort required for the user to perform input operations.

[0097] The information processing system (1) of the fifth aspect is the third or fourth aspect, and further includes a photographing unit (16) and a memory unit (12). The photographing unit (16) photographs the construction site of the target construction work. The memory unit (12) stores a composite image (A6) obtained by combining a photographed image (A3) photographed by the photographing unit (16) with a small blackboard (A5).

[0098] According to this aspect, the input work required by the user can be reduced when generating a composite image (A6) by combining a photographed image (A3) of the construction site of the target construction project with a small blackboard (A5).

[0099] The information processing system (1) of the sixth aspect is the information processing system of the fifth aspect, further comprising a display unit (15) that displays the composite image (A6).

[0100] According to this aspect, the user can check the composite image (A6) displayed on the display unit (15).

[0101] The information processing method of the seventh aspect includes an acquisition step, a voice recognition step, and a small blackboard selection step. In the acquisition step, voice data of a voice uttered by a user is acquired. In the voice recognition step, voice recognition processing is performed on the voice data. In the small blackboard selection step, a target form corresponding to a target task is selected from a plurality of small blackboard forms (A4) corresponding to a plurality of types of tasks, respectively. Each of the plurality of small blackboard forms (A4) includes a plurality of input items (D11 to D14) according to the type of task. In the voice recognition step, the voice data is voice recognized to recognize at least a portion of the plurality of input items (D11 to D14) and the plurality of pieces of input information entered into the plurality of input items (D11 to D14). In the small blackboard selection step, a target form is selected from the plurality of small blackboard forms (A4) based on the recognition result in the voice recognition step.

[0102] According to this aspect, it is possible to reduce the effort required for the user to perform input operations.

[0103] The program of the eighth aspect is a program for causing one or more processors of a computer system to execute the information processing method of the seventh aspect.

[0104] According to this aspect, it is possible to reduce the effort required for the user to perform input operations.

[0105] Not limited to the above aspects, various configurations (including modified examples) of the information processing system (1) according to the embodiment can be embodied as an information processing method performed by the information processing system (1), a (computer) program, or a non-transitory recording medium on which a program is recorded, etc.

[0106] The configurations according to the second to sixth aspects are not essential for the information processing system (1) and can be omitted as appropriate. [Explanation of symbols]

[0107] 1. Information Processing Systems 12 Storage section 13 Acquisition Department 14 Control section 15 Display section 16 Photography Department 111 Voice Recognition Unit 112 Small Blackboard Selection Section 113 Small Blackboard Creation Department A3 Photographed image A4 small blackboard style A5 small blackboard A6 composite image D11~D14 Input items

Claims

1. an acquisition unit that acquires voice data of a voice uttered by a user; a voice recognition unit that performs voice recognition processing on the voice data; a small blackboard selection unit that selects a target format corresponding to a target task from a plurality of small blackboard formats that respectively correspond to a plurality of types of tasks; each of the plurality of small blackboard forms includes a plurality of input items according to the type of work; the voice recognition unit recognizes the plurality of input items and at least a part of the plurality of pieces of input information input into the plurality of input items by performing voice recognition on the voice data; the small blackboard selection unit selects the target style from the plurality of small blackboard styles based on the recognition result of the voice recognition unit. Information processing system.

2. further comprising an operation unit that accepts an operation by the user; The small blackboard selection unit Selecting two or more candidate styles as candidates for the target style from the plurality of small blackboard styles based on the recognition result of the voice recognition unit; When the operation unit receives a selection operation by the user to select the target form from the two or more candidate forms, the operation unit selects the target form corresponding to the target task based on the selection operation. The information processing system according to claim 1 .

3. a small blackboard creation unit that creates a small blackboard by inputting the plurality of pieces of input information into the plurality of input items included in the target form, The information processing system according to claim 1 .

4. the voice recognition unit recognizes the plurality of pieces of input information input into the plurality of input items by voice recognition of the voice data; the small blackboard creation unit creates the small blackboard by inputting the plurality of pieces of input information into the plurality of input items based on the recognition result of the voice recognition unit. The information processing system according to claim 3 .

5. An imaging unit that photographs the construction site of the target work; and a storage unit that stores a composite image obtained by combining the image captured by the image capturing unit with the small blackboard. The information processing system according to claim 3 .

6. Further comprising a display unit that displays the composite image. The information processing system according to claim 5 .

7. an acquisition step of acquiring voice data of a voice uttered by a user; a voice recognition step of performing voice recognition processing on the voice data; a small blackboard selection step of selecting a target form corresponding to a target task from a plurality of small blackboard forms respectively corresponding to a plurality of types of tasks, each of the plurality of small blackboard forms includes a plurality of input items according to the type of work; In the voice recognition step, the voice data is voice-recognized to recognize at least a part of the plurality of input items and the plurality of pieces of input information input into the plurality of input items, respectively; In the small blackboard selection step, the target style is selected from the plurality of small blackboard styles based on the recognition result in the voice recognition step. Information processing methods.

8. 8. The information processing method according to claim 7, wherein the information processing method is executed by one or more processors of a computer system, program.

Images