Information processing method, information processing program, information processing apparatus, and information processing system

Abstract

This invention provides an information processing method, information processing program, information processing device, and information processing system that can improve the convenience of creating electronic whiteboards with new layouts. [Solution] The information processing method is performed to create an electronic whiteboard or an electronic whiteboard template that displays one or more items, and includes the steps of recognizing the items to be written and the positions of the items from a whiteboard image, and arranging an area on the electronic whiteboard to display items corresponding to the items to be written at a position corresponding to the position of the items to be written.

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing program, an information processing apparatus, and an information processing system.

Background Art

[0002] As described in Patent Document 1, a method of selecting a style according to the type of work from a plurality of styles of an electronic blackboard is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As a style of an electronic blackboard (also referred to as an electronic small blackboard or a construction electronic small blackboard), a layout that has not been prepared in advance may be required. An improvement in convenience when creating an electronic blackboard with a new layout is demanded.

[0005] An object of the present disclosure is to provide an information processing method, an information processing program, an information processing apparatus, and an information processing system that can improve convenience when creating an electronic blackboard with a new layout.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present disclosure is an information processing method executed to create an electronic blackboard that displays one or more items or a template of the electronic blackboard, and includes a step of recognizing a written item and a position of the written item from a blackboard image, and a step of arranging, on the electronic blackboard, a region for displaying an item corresponding to the written item at a position corresponding to the position of the written item.

[0007] (2) In the recognition step of the information processing method described in (1) above, the blackboard image may be input to a recognition model, and the recognition results of the listed items and the positions of the listed items may be obtained from the recognition model.

[0008] (3) In the information processing method described in (2) above, the recognition model is a large-scale language model (LLM) and may be configured to output the result of recognizing the name of the item as a key and the content of the item as a value when the blackboard image, the position of the item on the blackboard image, and an instruction to recognize the item on the blackboard image in key-value format are input as prompts.

[0009] (4) In the arrangement step of the information processing method described in any one of (1) to (3) above, a part having an area for displaying an item corresponding to the item recognized from the blackboard image in the recognition step may be extracted. The extracted part may be placed at a position corresponding to the position of the item recognized from the blackboard image in the recognition step.

[0010] An information processing program (5) according to one embodiment of the present disclosure is an information processing program to be executed by a processor to create an electronic whiteboard or a template for the electronic whiteboard that displays one or more items, and includes the steps of recognizing items to be written and the positions of the items to be written from a blackboard image, and arranging on the electronic whiteboard an area for displaying items corresponding to the items to be written at a position corresponding to the position of the items to be written.

[0011] An information processing device (6) according to one embodiment of the present disclosure is an information processing device for creating an electronic blackboard or a template for the electronic blackboard that displays one or more items, and includes a control unit that recognizes the items to be displayed and the positions of the items to be displayed from a blackboard image, and places an area on the electronic blackboard that displays the items corresponding to the items to be displayed at a position corresponding to the position of the items to be displayed.

[0012] An information processing system (7) according to one embodiment of the present disclosure is an information processing system for creating an electronic blackboard or a template for the electronic blackboard that displays one or more items, comprising: a recognition unit that recognizes items to be written and the positions of the items to be written from a blackboard image; and a placement unit that places an area on the electronic blackboard that displays an item corresponding to the item to be written at a position corresponding to the position of the item to be written. [Effects of the Invention]

[0013] According to an information processing method, information processing program, information processing apparatus, and information processing system according to one embodiment of this disclosure, the convenience of creating an electronic whiteboard with a new layout is enhanced. [Brief explanation of the drawing]

[0014] [Figure 1] This is a block diagram showing an example configuration of the information processing system related to this disclosure. [Figure 2] This flowchart shows an example of the procedure for the information processing method related to this disclosure. [Figure 3] This is an example of an image loading screen. [Figure 4] This is an example of a blackboard image. [Figure 5] This is an example of the elements included in a blackboard image. [Figure 6] This is an example of a blackboard layout screen. [Modes for carrying out the invention]

[0015] Photographs taken at construction sites and other work sites, showing a blackboard alongside the object being inspected or recorded, are saved as construction records. For example, a blackboard photographed with the object being inspected (also called a small construction blackboard or simply a small blackboard) has a section for recording inspection results. When photographing multiple objects being inspected using common inspection items, preparing a template (also called a template) in advance so that the blackboards photographed with the objects are written on in the same layout reduces the effort required to create the blackboards and minimizes errors in writing on them. In other words, preparing a blackboard template in advance improves the convenience of using blackboards for photography.

[0016] In this case, instead of preparing a physical blackboard, an image of a blackboard, known as an electronic blackboard, may be prepared and pasted onto an image of the subject being photographed. The electronic blackboard may include construction-use electronic blackboards or electronic blackboards. Even with electronic blackboards, preparing templates in advance improves the convenience of using them for photography. Electronic blackboard templates may differ from site to site, user to user, local government to local government to general contractor to general contractor to subcontractor

[0017] In this disclosure, an electronic whiteboard template refers to a template that pre-embeds frequently used characters, symbols, figures, or images that are typically displayed on an electronic whiteboard. By using such a template, users can efficiently input information to be displayed on the electronic whiteboard.

[0018] One possible method for creating an electronic whiteboard template in the comparative example is to draw vertical or horizontal lines in an editor to create frames for writing site names, shooting dates, or inspection results on the electronic whiteboard. However, in the method described in the comparative example, when arranging multiple writing frames, adjacent frames are separated by lines, so changing the position or size of one frame affects adjacent frames. As a result, the template creation process becomes complicated.

[0019] Hereinafter, as an information processing method according to the present disclosure, a method for easily creating a template of an electronic blackboard will be described. Also, a method for easily creating an electronic blackboard used for photographing from a template of the electronic blackboard will be described.

[0020] (Configuration example of information processing system 1) Hereinafter, a specific configuration example of the information processing system 1 will be described with reference to FIG. 1. The information processing system 1 includes an information processing apparatus 10 and a database 20. The information processing apparatus 10 and the database 20 are communicably connected via a network. The information processing apparatus 10 and the database 20 may be directly communicably connected without passing through a network.

[0021] <Information processing apparatus 10> The information processing apparatus 10 is used to create a template of an electronic blackboard or the electronic blackboard itself. The information processing apparatus 10 includes a control unit 11, a communication unit 12, a display unit 13, and an input unit 14.

[0022] The control unit 11 controls each component of the information processing apparatus 10. The control unit 11 may include one or more general-purpose processors or dedicated processors specialized for specific processing, etc., but is not limited thereto. The processor may include a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. The control unit 11 may include one or more dedicated circuits. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

[0023] The control unit 11 may include a storage unit for storing various information or programs for operating each component of the information processing device 10. The storage unit may store system programs or application programs. The storage unit may include, for example, semiconductor memory, magnetic memory, or optical memory. The storage unit may include an electromagnetic storage medium such as a magnetic disk. The storage unit may include a non-temporary computer-readable medium. The storage unit may function as, for example, main memory, auxiliary memory, or cache memory. The storage unit may function as the work memory of the control unit 11. The storage unit may be configured integrally with the control unit 11 or may be configured separately from the control unit 11.

[0024] The communication unit 12 may include a communication module configured to communicate with other devices such as the database 20. The communication module may be configured to communicate with other devices via a network. The communication module may also be configured to communicate directly with other devices without a network. The communication module may support wired or wireless communication standards. The communication module may support mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication module may support communication standards such as LAN (Local Area Network). The communication module is not limited to these and may support various communication standards. When communication is performed via a network, the network may be wired or wireless. The network may be configured in various ways, such as the Internet or an intranet.

[0025] The display unit 13 displays information or data based on instructions from the control unit 11. The display unit 13 may include various displays such as a liquid crystal display, an organic EL (Electro-Luminescence) or inorganic EL display, or an LED (Light Emitting Diode) display.

[0026] The input unit 14 receives input of user operations or information or data and transmits it to the control unit 11. The input unit 14 may include an input device that receives input from the user. The input device may include, for example, a touch sensor or a pointing device such as a mouse. The input device may be configured as a touch display integrated with the display unit 13. The input device may include physical keys. The input device may include an audio input device such as a microphone.

[0027] The information processing device 10 may be implemented as a computer such as a desktop PC (Personal Computer), notebook PC, or tablet PC. The information processing device 10 may also be implemented as a smartphone or tablet. The information processing device 10 may be configured to include at least one server. The information processing device 10 is not limited to these examples and may include various devices. The control unit 11 of the information processing device 10 may be implemented using a cloud service or in an on-premise environment.

[0028] <Database 20> Database 20 stores information necessary to create an electronic whiteboard template or the electronic whiteboard itself. It also stores information related to the construction project. This information may include information related to the construction management. This information may include the processes involved in the project, the scheduled dates for each process, or the progress status of each process. When Database 20 includes information related to the construction management, it is also called the construction management database. This information may include drawing data used in the construction or images of the whiteboard used for photographing the construction site.

[0029] Database 20 may include at least one server. Database 20 may include, for example, semiconductor memory, magnetic memory, or optical memory. Database 20 may include an electromagnetic storage medium such as a magnetic disk. Database 20 may include a communication module configured to communicate with other devices such as the information processing device 10.

[0030] Database 20 may be implemented using cloud services or in an on-premises environment.

[0031] <Other configurations> The electronic blackboard template may be created from an image of an actual blackboard. The information processing system 1 may be equipped with a camera that takes pictures of the actual blackboard. The information processing device 10 may be connected to the camera in a communicative manner and acquire an image of the actual blackboard from the camera.

[0032] (Example of operation of Information Processing System 1) The following describes an example of how the information processing device 10 in the information processing system 1 related to this disclosure can easily create an electronic whiteboard template. Hereinafter, the electronic whiteboard template will also be referred to as a blackboard template.

[0033] The control unit 11 of the information processing device 10 may execute an information processing method including the steps of the flowchart illustrated in Figure 2. The information processing method may be implemented as an information processing program to be executed by the control unit 11. The information processing program may be stored on a non-temporary computer-readable medium.

[0034] <Loading a blackboard image> The control unit 11 reads the blackboard image (step S1). The blackboard image is image data of a blackboard that will serve as a sample for the blackboard template. The control unit 11 may, for example, display the file selection screen shown in Figure 3 on the display unit 13 and allow the person responsible for creating the blackboard template to select the file of the blackboard image to be used as a sample. The blackboard image contains information necessary when taking construction photos. The information contained in the blackboard image may be classified into items. The items of information contained in the blackboard image are also called listed items. In the blackboard image, listed items may be described to include both the item name and the item content. Listed items may also be described to include only the item content without including the item name.

[0035] Figure 4 shows an example of a blackboard image read by the control unit 11. The blackboard image includes, for example, a project name field 51, a shooting date field 52, a miniature drawing field 53, an item field 54, or a free input field 55. The project name field 51, shooting date field 52, miniature drawing field 53, item field 54, or free input field 55 are also collectively referred to as description fields. The description fields correspond to the items to be described.

[0036] The "Project Name" column 51 is a field for entering a name to identify the project or site from which the photograph with the attached electronic whiteboard was taken. In Figure 4, the item name in the "Project Name" column 51 is "Site Name," but it may also be other names such as "Project Name" or "Building Name." In Figure 4, the item content in the "Project Name" column 51 is "(Tentative Name) Y Residence New Construction Project," but it is not limited to this and may be any other name.

[0037] The "Date Taken" column 52 is for recording the date the photograph with the electronic whiteboard attached was taken. In Figure 4, the item name for the "Date Taken" column 52 is "Date Taken," but it may also be named "Date Inspected," etc. The item content for the "Date Taken" column 52 in Figure 4 is the date the photograph was taken.

[0038] The miniature drawing section 53 is a section where images such as miniature drawings corresponding to the subject of a photograph with an attached electronic whiteboard can be pasted, or text, symbols, or figures can be drawn. A miniature drawing is, for example, a small-sized drawing created by cutting out only the part corresponding to the subject of the photograph from a construction drawing. The contents of the miniature drawing section 53 in Figure 4 are miniature drawings of reinforcement drawings, but are not limited to this and may include other drawings or images.

[0039] Item column 54 is a column for recording the contents of various items other than the project name column 51 or the date of photography column 52. The project name column 51 or the date of photography column 52 may be included in item column 54.

[0040] In Figure 4, among the item column 54, the item name in the column labeled "Type of Work" contains the content "Reinforcement Work". The content of the "Type of Work" column is not limited to "Reinforcement Work", but may include various other types of work such as "Earthwork" or "Structural Work".

[0041] In Figure 4, among the item column 54, the item name of the column labeled "Measurement Point" contains the item content "(X1-2, Y2-3)", which specifies the measurement point of the reinforcing bar as coordinates on the drawing. The item content of the "Measurement Point" column is not limited to this example and may take various forms.

[0042] In Figure 4, among the 54 item columns, the item name for the column labeled "Thickness" is "200," which represents the measured thickness of the concrete relative to the reinforcing steel. The item names for the columns labeled "Main Reinforcement" and "Distribution Reinforcement" are "D13@200," which represent the measured diameter and pitch of the main reinforcement and distribution reinforcement, respectively.

[0043] The free input field 55 is a field for freely entering notes and other information. In Figure 4, the items in the free input field 55 are "Upper Reinforcement" and "Lower Reinforcement," which represent the names of the reinforcing bars.

[0044] The blackboard image may include image data of an actual blackboard. If the information processing system 1 is equipped with a camera, the control unit 11 may acquire image data of an actual blackboard from the camera and read it as the blackboard image.

[0045] The blackboard image may include image data of the electronic blackboard alone, not attached to a construction photograph. The blackboard image may include image data of the electronic blackboard attached to a construction photograph. The blackboard image may include image data of a construction photograph with an electronic blackboard attached.

[0046] The control unit 11 may read blackboard data for displaying the electronic blackboard as an image, instead of a blackboard image. The blackboard data may include, for example, vector data of ruled lines that define the frames of the information to be written on the electronic blackboard. The blackboard data may also include, for example, the positions of the information to be written on the electronic blackboard and the text data to be displayed at those positions. The blackboard data is not limited to these examples and may include data in various formats, as long as it is data that can reproduce the image of the electronic blackboard. The control unit 11 may reproduce a blackboard image from the blackboard data and read the reproduced blackboard image.

[0047] <Recognition of listed items> Referring again to Figure 2, the control unit 11 recognizes the items written on the blackboard image (step S2). When the control unit 11 recognizes the items written on the blackboard image, it also recognizes the position of the items. The control unit 11 may consider the frames enclosed by ruled lines in the blackboard image as writing fields and recognize the corresponding items based on the content written in the writing fields. For example, when the control unit 11 recognizes the items written on the blackboard image in Figure 4, it may consider the frame labeled "Site Name" and the frame labeled "(Tentative Name) Y Residence New Construction Project" as a construction name field 51 in which the item name frame and the item content frame are connected, and recognize that the item is the site name. In other words, when the control unit 11 recognizes the writing on the blackboard image, it may not only recognize the content as simple text, but also recognize it as an item written on the blackboard image.

[0048] The control unit 11 may recognize the items written on the blackboard image using a recognition model. The recognition model is configured to output the recognition results of the items written on the blackboard image and the positions of those items when a blackboard image is input. The recognition model may be a trained model generated by performing training using training data that associates a blackboard image with the correct data of the items written on the blackboard image and the positions of those items.

[0049] The recognition model may be a Large-Scale Language Model (LLM). The LLM used as the recognition model is configured to output the result of recognizing the item names as keys and the item contents as values, when prompted with a blackboard image, the positions of the items contained in the blackboard image, and instructions to recognize the items from the blackboard image in key-value format. The prompts input to the LLM may be, but are not limited to, phrases such as, "You are a skilled construction worker. You are given an image of a blackboard used at a construction site. Read the table and output it in key-value format."

[0050] <<Extracting parts corresponding to the listed items>> The control unit 11 may extract the items recognized from the blackboard image as parts to be used on the electronic blackboard. Figure 5 shows examples of parts used on the electronic blackboard, including "item name + item content part," "item content only part," "mini-diagram part," and "table part." The parts used on the electronic blackboard are not limited to those exemplified in Figure 5 and may include various other forms.

[0051] The "item name + item content part" is a part that combines a cell where the text of the item name can be entered and a cell where the text of the item content can be entered. In Figure 5, the "item name + item content part" is configured so that the item name cell and the item content cell are arranged horizontally, but they may also be arranged vertically. The text of the item name may be text unique to the part, or it may be text that can be freely changed by the person in charge of creating the blackboard template, or the person in charge of creating an electronic blackboard using the blackboard template. The text of the item content may also be text that can be freely changed by the person in charge of creating the blackboard template, or the person in charge of creating an electronic blackboard using the blackboard template. The text of the item name or item content may be text that is transcribed from information that corresponds to the item name from information obtained from the construction management system, etc.

[0052] A "item content only part" is a part that has a cell on its own where the text of the item content can be entered. The text of the item content can be freely changed by the person in charge of creating the blackboard template, similar to the text of the item content in the "item name + item content part," or it can be text that has been copied from information obtained from a construction management system, etc.

[0053] A "dimensions component" is a component that has a cell on its own that can be used to paste images such as diagrams, or to draw text, symbols, or shapes. A "dimensions component" may also be a component that combines a cell on which text for item names can be entered with a cell on which images such as diagrams can be pasted or shapes can be drawn. Images such as diagrams may be images selected by the person in charge of creating the blackboard template, or the person in charge of creating an electronic blackboard using the blackboard template. Images such as diagrams may be pre-prepared images. Images such as diagrams may be images cut out from construction drawings by the person in charge of creating the blackboard template, or the person in charge of creating an electronic blackboard using the blackboard template. Cells on which drawings can be drawn may be configured so that shapes can be drawn in response to the drawing operations of the user, such as the person in charge of creating the blackboard template, or the person in charge of creating an electronic blackboard using the blackboard template. Cells on which images can be pasted and drawings can be drawn may be configured so that shapes can be drawn on the pasted image in response to the user's drawing operations.

[0054] A "table component" is a component formed by combining cells that can contain text, arranged in two or more rows and two or more columns. At least one cell in a "table component" may be a cell that can contain data other than text, such as a diagram, or a cell that can draw characters, symbols, or shapes. The text that can be entered in the cells of a "table component" may be freely changed by the person responsible for creating the blackboard template.

[0055] The control unit 11 may, for example, extract the project name field 51, the shooting date field 52, and the item field 54 from the blackboard image in Figure 4 as "item name + item content part", extract the miniature drawing field 53 as "miniature drawing part", and extract the free input field 55 as "item content only part".

[0056] The control unit 11 may identify the position and size of the parts extracted from the blackboard image. The control unit 11 may recognize the relative positional relationship and the relative size relationship of the extracted parts.

[0057] The control unit 11 may use a parts extraction model to extract parts from the blackboard image. The parts extraction model may be a trained model generated by performing training using training data that associates a blackboard image with ground truth data of the types, positions, and sizes of parts contained in the blackboard image.

[0058] <Creating a blackboard template> Referring again to Figure 2, the control unit 11 creates a blackboard template (step S3). Based on the items to be written and their positions recognized in step S3, the control unit 11 creates the blackboard template by arranging the areas for displaying items corresponding to the items on the electronic blackboard at positions corresponding to the items. The control unit 11 may draw grid lines to demarcate the areas for displaying items corresponding to the items.

[0059] The blackboard template may be configured so that when creating an electronic blackboard using the blackboard template, the creator of the electronic blackboard can input the contents of the items placed on the blackboard template. The blackboard template may also be configured so that information from a construction management system, etc., is reflected in the contents of the items placed on the blackboard template. When information from a construction management system, etc., is reflected in the contents of the items, tags or the like may be associated with the items on the blackboard template to identify the information to be reflected in the contents of the items.

[0060] When the control unit 11 writes the item names of the items to be described on the blackboard template, it may write the item names themselves that it recognized from the blackboard image.

[0061] The control unit 11 may replace the item name recognized from the blackboard image with a pre-registered name and write it to the blackboard template. The name to be used as an item name may be determined by each contractor or each site. Therefore, item names may be pre-registered for each contractor or site. For example, if the item name for the project name field 51 is written as "Site Name" in the blackboard image, and the pre-registered name is "Project Name", the control unit 11 may write "Project Name" instead of "Site Name" as the item name to the blackboard template.

[0062] The control unit 11 may set a style such as font, font size, or font color for each area to be placed on the blackboard template.

[0063] <<Creating a blackboard template by arranging parts>> If the control unit 11 extracts parts in step S3, it may, in the blackboard layout screen shown in Figure 6, select the parts corresponding to the parts extracted in step S3 from the parts selection frame 32, adjust the position and size of the parts, and automatically place them in the layout frame 31.

[0064] For example, the control unit 11 may select the project name part 41 from the part selection frame 32 as the part corresponding to the project name field 51 in the blackboard image. The control unit 11 may select the project name part 41 by reinterpreting the item name as "Project Name" because, although the item name of the project name field 51 in the blackboard image in Figure 4 is "Site Name," only parts with the item name "Project Name" can be selected from the part selection frame 32. The control unit 11 may adjust the vertical and horizontal size of the project name part 41 and place it in the layout frame 31 at the position corresponding to the project name field 51 in the blackboard image. The control unit 11 may adjust the size of the part while maintaining the aspect ratio of the parts that can be selected from the part selection frame 32, or it may adjust the vertical and horizontal size of the part independently.

[0065] The control unit 11 may select the shooting date part 42 from the part selection frame 32 as the part corresponding to the shooting date column 52 of the blackboard image. The control unit 11 may adjust the vertical and horizontal size of the shooting date part 42 and place it in the layout frame 31 at the position corresponding to the shooting date column 52 of the blackboard image.

[0066] The control unit 11 may select a miniature diagram part 43 from the part selection frame 32 as a part corresponding to the miniature diagram section 53 of the blackboard image. The control unit 11 may adjust the vertical and horizontal size of the miniature diagram part 43 and place it in the layout frame 31 at a position corresponding to the shooting date section 52 of the blackboard image.

[0067] The control unit 11 may select the item name part 44 from the part selection frame 32 as the part corresponding to the item column 54 of the blackboard image. In this example, assume that the part selection frame 32 does not contain any parts with item names corresponding to the item names "Work Type," "Measurement Point," "Thickness," "Main Reinforcement," and "Distributing Reinforcement" in the item column 54 of the blackboard image in Figure 4. In this case, since the control unit 11 cannot select a part with the item names of the item column 54 included in the blackboard image in Figure 4 from the part selection frame 32, it selects the item name part 44 as the part corresponding to the item column 54. The control unit 11 may adjust the vertical and horizontal size of the item name part 44 and place it in the layout frame 31 at the position corresponding to the item column 54 of the blackboard image.

[0068] The control unit 11 may select a free input part 45 from the part selection frame 32 as a part corresponding to the free input field 55 of the blackboard image. The control unit 11 may adjust the vertical and horizontal size of the free input part 45 and place it in the layout frame 31 at a position corresponding to the free input field 55 of the blackboard image.

[0069] The parts that can be selected from the parts selection frame 32 may have tags associated with their item contents. For example, the item contents of the shooting date part 42 may be associated with a tag that automatically inputs the shooting date. Also, the item contents of the construction name part 41 may be associated with a tag that automatically inputs the project name from the construction management system, etc.

[0070] <Blackboard template modification> Referring again to Figure 2, the control unit 11 modifies the blackboard template in response to the operation input from the person responsible for creating the blackboard template (step S4). The control unit 11 may change the position or size of the area that displays the items corresponding to the items written in the blackboard image in response to the operation input received by the input unit 14 from the person responsible for creating the template. The control unit 11 may change the position of the ruled lines in response to the change in the position or size of the area. The control unit 11 may change the item name in response to the operation input. The control unit 11 may change the tags, etc., associated with the item content in response to the operation input.

[0071] If parts are placed on the blackboard template, the control unit 11 may add, delete, move, or resize parts in response to operation input received by the input unit 14 from the creator. For example, in the blackboard layout screen of Figure 6, the input unit 14 may accept operation input such as deleting, moving, or resizing parts placed on the layout frame 31. The input unit 14 may accept operation input for selecting parts in the part selection frame 32 and operation input for placing the selected parts on the layout frame 31. The input unit 14 is not limited to these examples and may accept various operation inputs.

[0072] <Output of blackboard template> Referring again to Figure 2, the control unit 11 outputs a blackboard template (step S5). The control unit 11 may generate information combining the position and size of the parts placed on the blackboard template and the contents written on the parts, and output this information as blackboard template information. The control unit 11 may also output an image of the blackboard template. The control unit 11 may store the blackboard template in the database 20 or display it on the display unit 13.

[0073] The control unit 11 may output the blackboard template with the parts automatically arranged in step S3 without executing the procedure in step S4, or it may output the blackboard template modified in step S4. After executing the procedure in step S5, the control unit 11 terminates the execution of the flowchart in Figure 2.

[0074] <Creating an interactive whiteboard using a blackboard template> A blackboard template is used to create an electronic blackboard. The information processing device 10 may provide a blackboard template to a device used to create an electronic blackboard. The information processing device 10 may also provide a function to create an electronic blackboard from a blackboard template.

[0075] When the information processing device 10 creates an electronic blackboard, the control unit 11 may acquire information about the construction or work to be photographed using the electronic blackboard and select a blackboard template based on the acquired information. The control unit 11 may also accept input from the person responsible for creating the electronic blackboard at the input unit 14 and select a blackboard template.

[0076] The control unit 11 fills in the contents of the items to be written in the selected blackboard template. If tags or the like are associated with the items to be written in the blackboard template, the control unit 11 may obtain the information identified by the tags or the like from the construction management system or the like and reflect it in the item contents. The control unit 11 may receive input from the person in charge of creating the electronic blackboard, etc., at the input unit 14 and reflect the input content in the item contents.

[0077] The control unit 11 may generate and output an image of an electronic blackboard with the contents of the items in the blackboard template filled in, so that it can be attached to the construction photograph. The control unit 11 may store the generated electronic blackboard image in the database 20 or the like, or display it on the display unit 13.

[0078] (summary) As described above, the information processing system 1, information processing device 10, information processing method, and information processing program according to this embodiment allow for the easy generation of a blackboard template from a blackboard image. As a result, the convenience of creating an electronic blackboard with a new layout is improved.

[0079] While embodiments relating to this disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are included within the scope of this disclosure. For example, the functions included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided. While embodiments relating to this disclosure have been described primarily in terms of apparatus, embodiments relating to this disclosure can also be realized as methods including steps performed by each component of the apparatus. Embodiments relating to this disclosure can also be realized as methods, programs, or storage media recording programs executed by a processor in the apparatus. These should also be understood to be included within the scope of this disclosure.

[0080] For example, at least some of the functions of the information processing device 10 relating to this disclosure may be shared among multiple servers, or shared among multiple terminal devices, or shared among one or more servers and one or more terminal devices. In the information processing system 1, for example, a recognition unit that recognizes the items to be written and the positions of the items to be written from a blackboard image, and a placement unit that places an area for displaying the items corresponding to the items to be written on the electronic blackboard at a position corresponding to the position of the items to be written, may be shared among different servers or terminal devices. [Explanation of Symbols]

[0081] 1. Information Processing System 10. Information processing device (11: control unit, 12: communication unit, 13: display unit, 14: input unit) 20 Databases 31 Layout Frames 32 parts selection slots 37. Complete button 38 File Selection Button 41 Construction Name Parts 42 Shooting date parts 43 Miniature Diagram Parts 44 Item Name Parts 45 Free Input Parts 51. Project Name Column 52 Date of shooting column 53 Map column 54 Item column 55 Free input field

Claims

1. An information processing method performed to create an electronic whiteboard or a template for the electronic whiteboard that displays one or more items, A step of recognizing the items to be written and the location of those items from a blackboard image, The electronic whiteboard includes the steps of arranging an area for displaying items corresponding to the items to be written at a position corresponding to the position of the items to be written. Information processing methods, including those mentioned above.

2. The information processing method according to claim 1, wherein in the recognition step, the blackboard image is input to a recognition model, and the recognition results of the listed items and the positions of the listed items output from the recognition model are obtained.

3. The information processing method according to claim 2, wherein the recognition model is a large-scale language model (LLM), and is configured to output the result of recognizing the name of the item as a key and the content of the item as a value when the blackboard image, the position of the item on the blackboard image, and an instruction to recognize the item on the blackboard image in key-value format are input as prompts, the recognition model is a large-scale language model (LLM), and the recognition model is configured to recognize the name of the item as a key and the content of the item as a value.

4. In the aforementioned arrangement step, In the recognition step described above, a part having an area for displaying items corresponding to the items recognized from the blackboard image is extracted. The information processing method according to any one of claims 1 to 3, wherein the extracted parts are placed at positions corresponding to the positions of the written items recognized from the blackboard image in the recognition step.

5. An information processing program that a processor is instructed to execute in order to create an electronic whiteboard that displays one or more items or a template for said electronic whiteboard, A step of recognizing the items to be written and the location of those items from a blackboard image, The electronic whiteboard includes the steps of arranging an area for displaying items corresponding to the items to be written at a position corresponding to the position of the items to be written. Information processing programs, including those mentioned above.

6. An information processing device for creating an electronic whiteboard that displays one or more items or a template for said electronic whiteboard, An information processing device comprising a control unit that recognizes items to be written and the positions of those items from a blackboard image, and places an area on the electronic blackboard that displays items corresponding to those items at a position corresponding to the positions of those items.

7. An information processing system for creating an electronic whiteboard that displays one or more items, or a template for said electronic whiteboard, A recognition unit that recognizes the items to be written and the positions of those items from a blackboard image, The electronic whiteboard includes an arrangement section which arranges an area for displaying items corresponding to the items to be written at a position corresponding to the position of the items to be written. An information processing system equipped with the following features.

Images