Programs, methods, information processing devices, systems
The system automates construction photo management by presenting high-probability tags and using image analysis to reduce worker burden and enhance efficiency in image tagging and association.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- OPTIM
- Filing Date
- 2023-11-22
- Publication Date
- 2026-05-29
Smart Images
Figure 0007867316000001 
Figure 0007867316000002 
Figure 0007867316000003
Abstract
Description
Technical Field
[0004] ,
[0006] , , , , ,
[0005] , , , , ,
[0001] The present disclosure relates to a program, a method, an information processing apparatus, and a system.
Background Art
[0002] A construction photo management method for reducing the burden on workers has been proposed (see Patent Document 1). In Patent Document 1, a construction photo management system creates a tag composed of a one-dimensional or two-dimensional code and incorporates it into an image, so that necessary construction data can be automatically read by an information processing unit.
Prior Art Document
Patent Document
[0003] This is a program to be executed by a computer equipped with a processor and memory. The program causes the processor to perform the following steps: extract candidate tags from multiple tags, any of which contain characters related to the subject being photographed, based on information about the photograph; present the extracted candidate tags in a selectable format; accept user selections for the presented candidate tags; photograph the subject being photographed; and store the photographed image, associating it with the tags specified by the user. [Effects of the Invention]
[0007] According to this disclosure, the burden on workers when acquiring images with excellent manageability can be further reduced. [Brief explanation of the drawing]
[0008] [Figure 1] This is a block diagram showing an example of the overall configuration of System 1. [Figure 2] Figure 1 is a block diagram showing an example configuration of the terminal device 10. [Figure 3] This figure shows an example of the functional configuration of server 20. [Figure 4] This diagram shows the data structure of user information 181. [Figure 5] This diagram shows the data structure of tag information 182. [Figure 6] This figure shows the data structure of related information 183. [Figure 7] This figure shows the data structure of image information 184. [Figure 8] This diagram shows the data structure of report information 185. [Figure 9] This is a flowchart illustrating an example of the operation of terminal device 10 when storing image data. [Figure 10] This is a schematic diagram showing an example of the display of the terminal device 10 when storing images necessary for diagnosis. [Figure 11] This is a flowchart illustrating an example of the operation of terminal device 10 when creating a report. [Figure 12]It is a schematic diagram showing an example of the display of the terminal device 10 when displaying a list of reports. [Figure 13] It is a schematic diagram showing an example of the display of the report creation screen. [Figure 14] It is a flowchart showing an example of other operations of the terminal device 10 when storing image data. [Figure 15] It is a block diagram showing the basic hardware configuration of the computer 90.
Mode for Carrying Out the Invention
[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.
[0010] <Summary> The terminal device installed with the program according to this embodiment presents a plurality of tags with high probability given to the image data to the user. The terminal device stores the captured image in association with the tag selected by the user.
[0011] When the user creates a report, when an image attached to the report is selected by the user, the terminal device reads out the tag attached to the selected image and inputs the characters related to the tag into the report in an editable manner. The user creates a report using the characters related to the tag.
[0012] <1 Configuration Diagram of the Whole System> FIG. 1 is a block diagram showing an example of the overall configuration of the system 1. The system 1 shown in FIG. 1 includes, for example, a terminal device 10 and a server 20. The terminal device 10 and the server 20 are communicatively connected via, for example, a network 80.
[0013] In FIG. 1, an example in which the system 1 includes one terminal device 10 is shown, but the number of terminal devices 10 included in the system 1 is not limited to one. The terminal device 10 is, for example, a terminal possessed by a user. The number of terminal devices 10 included in the system 1 may be two or more.
[0014] In FIG. 1, an example in which the system 1 includes the server 20 is shown, but the servers included in the system 1 are not limited to these. For example, servers other than the server 20 may be included.
[0015] In the present embodiment, an aggregate of a plurality of devices may be regarded as one server. The way of distributing the plurality of functions required to implement the server 20 according to the present embodiment for one or more pieces of hardware can be appropriately determined in view of the processing capabilities of each piece of hardware and / or specifications required for the server 20.
[0016] The terminal device 10 shown in FIG. 1 is an information processing device operated by a user. In the present embodiment, the user is, for example, a person who performs a predetermined diagnostic task. In the present embodiment, the predetermined diagnostic task includes, for example, a task related to house diagnosis or a task related to diagnosis for farmers' insurance. The terminal device 10 is realized by, for example, a mobile terminal such as a smartphone or a tablet having a camera function. If the terminal device 10 has a camera function, it may be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch-type terminal. Note that the terminal device 10 may be a stationary PC (Personal Computer) or a laptop PC.
[0017] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The input device 13 is a device for receiving an input operation from a user (for example, a pointing device such as a touch panel, a touch pad, a mouse, etc., a keyboard, etc.). The output device 14 is a device for presenting information to the user (display, speaker, etc.).
[0018] Server 20 is, for example, an information processing device that manages tags assigned to image data. Server 20 is implemented, for example, by a computer connected to network 80. As shown in Figure 1, Server 20 includes a communication IF 22, an I / O IF 23, memory 25, storage 26, and a processor 29. The I / O IF 23 functions as an interface to an input device for receiving input operations from the user and an output device for outputting information to the user.
[0019] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10 and the server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.
[0020] <1.1 Terminal Device Configuration> Figure 2 is a block diagram showing an example configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 160, a location information sensor 150, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.
[0021] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.
[0022] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 on which instructions are input by touching the operating surface. If the terminal device 10 is a PC, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.
[0023] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.
[0024] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.
[0025] Camera 160 is a device that receives light using a photodetector and outputs it as image data.
[0026] The location information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10, which is equipped with a GPS module, is detected based on the received signals. The location information sensor 150 may also detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected. The location information sensor 150 may also obtain the current position of the terminal device 10 using beacon signals. Alternatively, the location information sensor 150 may obtain the current position of the terminal device 10 using its IP address. The current position obtained by the location information sensor 150 may be the coordinates or address of the terminal device 10's current location, or the name of the facility where the terminal device 10 is currently staying.
[0027] The storage unit 180 is implemented by, for example, memory 15 and storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181, tag information 182, related information 183, image information 184, and report information 185.
[0028] User information 181 stores, for example, personal information about the user using terminal device 10. Further details will be provided later. Tag information 182 stores information about the tag being remembered, for example. More details will be provided later. Related information 183, for example, stores the relationships between tags. More details will be provided later. Image information 184 stores information about the captured image, for example. Further details will be provided later. Report information 185 stores information related to the report, for example. Further details will be provided later.
[0029] The control unit 190 is realized when the processor 19 reads a program stored in the memory unit 180 and executes instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 performs functions such as an operation reception unit 191, a transmission / reception unit 192, a management unit 193, a shooting control unit 194, a creation unit 195, and a presentation control unit 196.
[0030] The operation reception unit 191 processes instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives instructions or information input from a touch-sensitive device 131 or the like.
[0031] Furthermore, the operation reception unit 191 receives voice instructions input from the microphone 171. Specifically, for example, the operation reception unit 191 receives voice signals input from the microphone 171 and converted into digital signals by the voice processing unit 17. The operation reception unit 191 obtains instructions from the user by, for example, analyzing the received voice signals and extracting predetermined nouns.
[0032] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 sends instructions input by the user to the server 20. The transmitting / receiving unit 192 also receives information provided by the server 20.
[0033] The management unit 193 manages the tag information stored in the tag information 182. Specifically, for example, the management unit 193 manages the tag information so that the tag information stored in the tag information 182 is up-to-date. More specifically, for example, the management unit 193 checks with the server 20 at predetermined intervals to see if there have been any updates to the tag information. The predetermined interval may be, for example, once a day, once every few days, or once a predetermined week. If there has been an update to the tag information, such as an increase or decrease in the number of tags, the server 20 transmits the tag information stored in the server 20 to the terminal device 10. The management unit 193 receives the information transmitted from the server 20 and updates the information stored in the tag information 182.
[0034] Server 20 may transmit information about the tags stored in Server 20 to the terminal device 10 at predetermined intervals. Server 20 may transmit information only when there is an update, or it may transmit information even when there is no update.
[0035] The shooting control unit 194 controls the storage of images captured by the camera 160 of the terminal device 10. Specifically, for example, the shooting control unit 194 extracts tags suitable for the subject to be photographed by the user from the tags stored in the tag information 182 as candidate tags. The shooting control unit 194 extracts tags that satisfy predetermined requirements related to photography from a plurality of tags stored in the tag information 182 as candidate tags. Tags that satisfy predetermined requirements include, for example, the following. These requirements may be adopted independently, or at least two may be adopted in combination. • Tags associated with the management group that manages the acquired image data. • Tags associated with specific attributes included in the user's personal information • Tags associated with the subject being photographed • Tags associated with the subject recognized through image analysis • Tags associated with the photographed object, identified by decrypting the code related to the photographed object (for example, the QR code (registered trademark) attached to the photographed object). • Tags associated with situations that are understood based on the user's schedule. • Tags associated with the situation as determined from the user's current location
[0036] Image analysis may be performed, for example, by an image analysis model. The image analysis model is a trained model generated by having a machine learning model perform machine learning according to a model learning program. More specifically, the image analysis model is generated, for example, using a feedforward multilayer network. As the multilayer network, for example, a deep neural network (DNN), which is a multilayer neural network targeted by deep learning, can be used. Note that the image analysis model may also be generated using existing technologies. The image capture control unit 194 estimates the devices contained in the image by inputting the image to the image analysis model, for example. The estimated information is, for example, the name of the device, the identification information of the device, etc.
[0037] The shooting control unit 194 may, for example, create tags based on the user's current state and extract candidate tags from the tags stored in the tag information 182 and the created tags. The user's current state includes, for example, the user's schedule. The schedule may be stored in, for example, user information 181 or managed on a predetermined server. The shooting control unit 194 may, for example, create tags using the current time and corresponding wording in the schedule. Specifically, for example, if the schedule is set as "14:00 to 15:00: Diagnose BBBB at factory AAAA" and the time is 14:00, the shooting control unit 194 will create tags related to factory AAAA and tags related to BBBB, which is the target of the diagnosis.
[0038] The user's current state includes, for example, the user's current location. The user's current location is acquired, for example, by the location information sensor 150. The image capture control unit 194 creates a tag using text related to the acquired current location. Specifically, for example, if the acquired current location is "Facility: Factory AAAA", the image capture control unit 194 creates a tag related to Factory AAAA. Also, for example, if the acquired current location is information that does not directly represent a facility, such as "Coordinates: (X, Y)" or "Address: CCCC", the image capture control unit 194 acquires the facility associated with "Coordinates: (X, Y)" or "Address: CCCC" and creates a tag related to the acquired facility.
[0039] The shooting control unit 194, for example, receives a user's specification for extracted candidate tags. The shooting control unit 194 controls the camera 160 to photograph the target object. In this embodiment, photographing the target object is not limited to, for example, storing the image data generated by the camera 160 in the storage unit 180. In other words, photographing is not limited to storing image data executed by pressing the shooting button, but also includes displaying the image obtained by the camera 160 on the display 141.
[0040] The shooting control unit 194 stores the image data generated by the camera 160 in the image information 184, for example, in response to the pressing of the shooting button. At this time, the shooting control unit 194 associates the image data with a tag specified by the user.
[0041] If the user does not specify which candidate tags to use, the shooting control unit 194 may, for example, identify the subject to be photographed and obtain the tags associated with the identified subject from the candidate tags. The shooting control unit 194 may also associate the obtained tags with the image data stored in the image information 184. Specifically, the shooting control unit 194 identifies the subject being photographed, for example, by image analysis. Furthermore, if the image contains a code, the shooting control unit 194 decodes the code to obtain information that identifies the subject to be photographed and identifies the subject based on that information.
[0042] If the shooting control unit 194 associates a tag with image data by recognizing the object to be photographed, it may notify the user that a tag has been associated without the user's explicit request. The shooting control unit 194 may, in response to user instructions, replace the tag associated with the image data after storing the image data. For example, the shooting control unit 194 stores that the tag has been replaced. For example, the shooting control unit 194 may improve the accuracy of the automatic tagging process based on the information that the tag has been replaced.
[0043] The creation unit 195 creates a report. Specifically, for example, the creation unit 195 creates a report on a predetermined diagnosis in response to instructions from the user. The creation unit 195 accepts the specification of images to be attached to the report. The creation unit 195 editably enters the characters related to the tags associated with the specified images into the report's input form. The creation unit 195 accepts edits from the user and creates the report.
[0044] The presentation control unit 196 controls the output device 14 to present predetermined information to the user. Specifically, for example, the presentation control unit 196 displays the image of the subject being photographed on the display 141. The presentation control unit 196 also displays candidate tags extracted by the shooting control unit 194 on the display 141. At this time, the presentation control unit 196 displays the candidate tags based on the relationships between the tags.
[0045] Furthermore, the display control unit 196 displays images on the display 141, for example, when creating a report.
[0046] <1.2 Functional Configuration of the Server> Figure 3 shows an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.
[0047] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.
[0048] The storage unit 202 includes, for example, a tag table 2021. However, the tables stored in the storage unit 202 are not limited to this.
[0049] Tag Table 2021 is a table that stores information about tags. Tag Table 2021 is a table that uses Tag ID as the key and has columns for tags (content), related attributes, related devices, related equipment, related groups, etc.
[0050] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs functions such as the receive control module 2031, the transmit control module 2032, and the management module 2033.
[0051] The receive control module 2031 controls the process by which the server 20 receives signals from external devices according to a communication protocol.
[0052] The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.
[0053] The management module 2033 manages the information of the tables stored in the storage unit 202. Specifically, for example, the management module 2033 manages the information about tags stored in the tag table 2021.
[0054] <2 Data Structure> Figures 4 to 8 show the data structure of the information stored by the terminal device 10. Note that Figures 4 to 8 are examples and do not exclude data not shown. Furthermore, even data included in the same information may be stored in separate memory areas within the storage unit 180.
[0055] Figure 4 shows the data structure of user information 181. User information 181 shown in Figure 4 has columns such as name, age, gender, date of birth, contact information, and affiliation, with user ID as the key.
[0056] The User ID field stores an identifier to uniquely identify the user. The Name field stores the user's name. The Age field stores the user's age. The Gender field stores the user's gender. The Date of Birth field stores the user's date of birth. The Contact Information field stores the user's contact information. The Affiliation field stores the user's affiliation. The user's affiliation represents, for example, the attributes involved in a specific diagnostic task. For example, the "Affiliation" field includes departments, sections, etc., of a specific company involved in home inspections. Also, for example, the "Affiliation" field includes departments, sections, etc., of a specific company involved in inspections for farmers' insurance.
[0057] Figure 5 shows the data structure of tag information 182. The tag information 182 shown in Figure 5 has columns such as tag, related attribute, related device, related equipment, and related group, with tag ID as the key.
[0058] The tag ID is an item that stores an identifier to uniquely identify a tag. The tag is an item that stores the content of the tag. For example, the "tag" item stores text information that represents the content of the tag. In the example shown in Figure 5, the "tag" item stores a tag that contains text related to the area being diagnosed. The "tag" item also stores a tag that contains text related to the equipment used in the diagnosis. The "tag" item also stores a tag that contains text related to the parts used in the equipment.
[0059] Area-related tags may include, for example, the following: • Text concerning the equipment or facilities subject to diagnosis • Text about the area where the facilities will be constructed • Text describing the processes being performed at the equipment being diagnosed. • Text concerning the equipment or facility in which the device subject to diagnosis is used.
[0060] Area-related tags include, for example, ash treatment (high-concentration treatment facility), ash treatment (low-concentration treatment facility), ash treatment (fly ash treatment facility), etc.
[0061] Tags associated with the device may include, for example, the following: • Text regarding the equipment or devices used in the facilities or equipment subject to diagnosis. • Text about the devices subject to diagnosis • Text about the equipment used in diagnosis
[0062] Tags related to equipment include, for example, ammonia injection equipment, ammonia extraction equipment, nitrogen oxide treatment equipment, hydrogen oxide treatment equipment, oxide treatment equipment, ampere breaker, etc. Tags related to parts include, for example, text relating to the name of a given part.
[0063] The "tags" field contains text that can be used as a document, for example, when creating a report. For example, a report might have a section for reporting the details of a diagnosis. This section might contain, for example, a report on a diagnosis performed at a facility or equipment. The "tags" field also stores, for example, words related to the diagnosis that are used when explaining the diagnosis.
[0064] Related attributes are fields that store the user attributes associated with a tag. Specifically, depending on the user's affiliation, the equipment or facilities to be diagnosed may be defined. Also, depending on the user's affiliation, the devices used in the diagnosis or the devices to be diagnosed may be defined. Furthermore, depending on the user's affiliation, the parts used in the device may be defined. The "Related Attributes" field stores the user attributes associated with each tag.
[0065] The "Associated Devices" field stores information about devices associated with a tag. For example, the "Associated Devices" field stores information about devices used in the diagnosis or devices being diagnosed. Depending on the device, the equipment or facilities used may be specified. Additionally, some devices may have specified related devices. Furthermore, some devices may have specified parts used. The "Associated Devices" field stores information about devices associated with each tag.
[0066] The "Related Equipment" field stores the area (equipment or facility) associated with the tag. For example, the "Related Equipment" field stores the area where the diagnosis is performed. Depending on the area, the equipment or facility to be diagnosed may be specified. Also, depending on the area, the devices used in the diagnosis, or the devices to be diagnosed, may be specified. Furthermore, depending on the area, the parts used in the devices may be specified. The "Related Equipment" field stores the area associated with each tag.
[0067] The "Management Group" field stores the management group that manages the captured images. A management group is, for example, a group used to manage information acquired in a related diagnosis. Image data managed by a management group is associated with similar tags. For example, when a user starts a diagnosis, they set a group to manage the acquired information. Image data acquired during the diagnosis is then managed, for example, by the configured management group. The "Management Group" field stores the management group that each tag is most likely to be associated with.
[0068] Figure 6 shows the data structure of related information 183. The related information 183 shown in Figure 6 stores the relationships between tags. Specifically, for example, the tags have a tree structure, and each element has multiple child elements. More specifically, the tags have a three-tiered tree structure, with "device" being a child element of the element related to "area," and "part" being a child element of the element related to "device."
[0069] Figure 7 shows the data structure of image information 184. The image information 184 shown in Figure 7 has columns such as image ID as the key, shooting date and time, tag, photographer, and image data.
[0070] The Image ID is an item that stores an identifier to uniquely identify the image data. The Date and Time of Capture is an item that stores the date and time the image data was stored. The Tag is an item that stores the tags associated with the image data. In the example shown in Figure 7, multiple tags are associated with each image data. Note that an image data may be associated with only one tag. The information stored in the "Tag" item may be the text information of the tag or the tag ID. The Photographer is an item that stores the user who took the image. For example, the "Photographer" item stores the user ID of the user who took the image. Note that the "Photographer" item may also store the name of the user who took the image. The Image Data is an item that stores the image data itself. The "Image Data" item may also store reference information to an image data file located elsewhere.
[0071] Figure 8 shows the data structure of report information 185. Report information 185 shown in Figure 8 is a table that has columns such as report name, author, creation date, and report data, with report ID as the key.
[0072] The Report ID is an item that stores an identifier to uniquely identify the report data. The Report Name is an item that stores the name of the report. The Report Name may be set by the user, for example, or it may be set automatically based on the diagnostic content. The Creator is an item that stores the user who created the report. The "Creator" item stores, for example, the user ID of the user who created the report. Alternatively, the "Creator" item may store the name of the user who created the report. The Creation Date is an item that stores the date and time the report was created. The Report Data is an item that stores the report data itself. The "Report Data" item may store reference information to a report data file located elsewhere.
[0073] <3 operations> (Image recording and processing) This section describes the processing of the terminal device 10 when a user operates the terminal device 10 during a prescribed diagnostic procedure and stores images necessary for the diagnosis.
[0074] First, when a user starts operating terminal device 10, for example, they operate terminal device 10 and enter their login ID and password. When a user starts a diagnosis, for example, they operate terminal device 10 and select a management group that will manage the information obtained from the diagnosis being performed.
[0075] Figure 9 is a flowchart illustrating an example of the operation of the terminal device 10 when storing image data. When the user needs to store an image of a specific object while performing a predetermined diagnosis, the user operates the terminal device 10 to photograph the object. For example, the user instructs the terminal device 10 to extract candidate tags on the shooting screen.
[0076] In step S11, the terminal device 10 acquires information related to the diagnosis. Specifically, for example, when candidate tag extraction is instructed, the shooting control unit 194 reads the user's affiliation stored in the user information 181 as a related attribute. The shooting control unit 194 also acquires the user's schedule and, based on the acquired schedule, acquires the currently running appointments. The shooting control unit 194 acquires the equipment or facilities, etc. included in the acquired appointments as related equipment. The shooting control unit 194 also acquires the equipment or facilities, etc. where the user is currently located as related equipment, based on the current location acquired by the location information sensor 150. The shooting control unit 194 also recognizes the device to be photographed and identifies the device. The shooting control unit 194 acquires the identified device as related equipment.
[0077] The diagnostic information acquired by the imaging control unit 194 may be only a part of the above, or other different information may be added.
[0078] In step S12, the terminal device 10 extracts candidate tags. Specifically, for example, the imaging control unit 194 extracts candidate tags from a plurality of tags stored in the tag information 182 based on the configured management group, acquired diagnostic information, or a combination thereof. More specifically, the imaging control unit 194 extracts tags from the tag information 182 in which, for example, a management group, related attributes, related devices, related equipment, or a combination of at least two of these is stored in the record.
[0079] This will extract tags such as the following as candidate tags. • Tags associated with the group that manages the acquired image data. • Tags associated with specific attributes included in the user's personal information • Tags associated with the device being photographed.
[0080] In step S13, the terminal device 10 presents the extracted candidate tags to the user. Specifically, for example, the presentation control unit 196 displays the extracted candidate tags on the display 141 so that the user can select them. The user selects a desired candidate tag from the multiple candidate tags displayed on the display 141.
[0081] In step S14, the terminal device 10 accepts the user's selection of candidate tags. Specifically, for example, the shooting control unit 194 accepts the user's selection instruction for multiple candidate tags displayed on the display 141. The user selects, for example, several tags that are expected to contribute to extracting image data. The user also selects, for example, several tags that are expected to be useful when creating a report. There is no limit to the number of tags that can be selected. Once the user has selected tags, they press a predetermined icon to switch the display to a screen for inputting instructions to store the image.
[0082] Figure 10 is a schematic diagram showing an example of the display of the terminal device 10 when storing images necessary for diagnosis. In the example shown on the left of Figure 10, the display control unit 196 displays a first display area 1411, a button 1412, and an icon 1413 on the display 141. The first display area 1411 is an area for displaying the subject being photographed. The button 1412 is a button for inputting an instruction to store the captured content as image data. The icon 1413 is an icon for inputting an instruction to switch the display to a screen for selecting tags. In other words, the icon 1413 is an icon for instructing the extraction of candidate tags. The icon 1413 is, for example, shaped to resemble the shape of a tag. When the icon 1413 is pressed by the user, the display control unit 196 switches the display to the right of Figure 10.
[0083] In the example shown in Figure 10 (right), the display control unit 196 causes the display 141 to display a first display area 1411 and a second display area 1414. The second display area 1414 is an area for displaying candidate tags. The display control unit 196 displays candidate tags in the second display area 1414 so that the user can select them. When a candidate tag displayed in the second display area 1414 is selected by the user, the display control unit 196 changes the appearance of the selected candidate tag so that the user can understand that it has been selected. The display control unit 196 displays the selected tag from the candidate tags in the first display area 1411. The display control unit 196 may also add an indicator to the tag displayed in the first display area 1411 that allows the selection to be canceled. Once the user has finished selecting a tag, they input a predetermined operation and switch the display on the display 141 from Figure 10 (right) to Figure 10 (left). When the display is switched to the left side of Figure 10 after a tag has been selected, the display control unit 196 may maintain the display of the tag shown in the first display area 1411 in the right side of Figure 10.
[0084] Figure 10 illustrates an example in which candidate tags are displayed in a row in the second display area 1414. However, the display of candidate tags is not limited to this. The presentation control unit 196 may, for example, display candidate tags in a manner that takes into account the relationships between tags. The presentation control unit 196 may, for example, display tags starting with those assigned a higher hierarchy, and when a higher-level tag is selected, display lower-level tags. Specifically, for example, the presentation control unit 196 first displays candidate tags whose element is "area" on the display 141. When a candidate tag whose element is "area" is selected, the presentation control unit 196 displays candidate tags for "devices" belonging to the selected candidate tag on the display 141. When a candidate tag whose element is "device" is selected, the presentation control unit 196 displays candidate tags for "parts" belonging to the selected candidate tag on the display 141.
[0085] In step S15, the terminal device 10 receives an instruction to store the captured content. The user, for example, selects a tag and switches the display on the display 141 to the left in Figure 10, then presses button 1412. When button 1412 is pressed, the shooting control unit 194 determines that the user has input an instruction to store the image. The image to be stored may be a still image or a video.
[0086] In step S16, the terminal device 10 stores image data. Specifically, when button 1412 is pressed, the display control unit 196 stores the image that was captured at the time button 1412 was pressed in image information 184, associating it with the selected tag.
[0087] In step S12, the shooting control unit 194 may create tags based on, for example, the user's current state. Specifically, when the user instructs the shooting control unit 194 to extract candidate tags, it may create tags based on, for example, wording included in the user's schedule. The shooting control unit 194 may also create tags based on, for example, wording related to the user's current location. The shooting control unit 194 extracts candidate tags from the tags stored in the tag information 182 and the tags it has created. The shooting control unit 194 may also create tags in response to the user's login.
[0088] If, for example, no tag is specified in step S14 and a storage instruction is input by the user in step S15, the shooting control unit 194 may acquire a tag from candidate tags based on various information and associate the acquired tag with the stored image data. Specifically, for example, the shooting control unit 194 identifies the target to be photographed by image analysis or based on a predetermined code attached to the target. The shooting control unit 194 acquires a tag that is stored as an associated device for the identified target to be photographed. The shooting control unit 194 also grasps the user's current state, for example, based on the current location or schedule. The shooting control unit 194 acquires a tag that is stored as an associated device for the grasped state.
[0089] When the image capture control unit 194 automatically associates a tag with image data (without the user selecting the tag), it notifies the user that the tag has been automatically associated. If the user requests a correction to the automatically associated tag, the image capture control unit 194 may replace the tag after storing the image data.
[0090] (Report creation process) This section describes the processing performed by the terminal device 10 when a user operates the terminal device 10 and creates a diagnostic report during a prescribed diagnostic procedure.
[0091] Figure 11 is a flowchart illustrating an example of the operation of the terminal device 10 when creating a report. The user creates a report once the predetermined diagnosis is complete. The user operates the terminal device 10 to display images related to the report.
[0092] Figure 12 is a schematic diagram showing an example of the display of the terminal device 10 when displaying a list of reports. In the example shown in Figure 12, the display control unit 196 displays a list of created reports on the display 141. The user can check the contents of a report by tapping on the report displayed in the list. The user can also edit a report by tapping on the report displayed in the list.
[0093] In the example shown in Figure 12, the display control unit 196 displays button 1415. Button 1415 is a button for inputting the instruction to start creating a report. By pressing button 1415, the user instructs the terminal device 10 to create a report.
[0094] In step S21, the terminal device 10 presents the user with a report creation screen. Specifically, for example, when the creation unit 195 receives a report creation instruction from the user, it presents the user with a report creation screen. The creation unit 195 then creates a report on a predetermined diagnosis in response to instructions from the user on the creation screen.
[0095] Figure 13 is a schematic diagram showing an example of the report creation screen. In the example shown in Figure 13(a), the report is designed to input information about the pre-work and post-work phases. However, the content of the report is not limited to this; it may also be a format that only inputs information about the post-work phase, or a format that inputs information about the pre-work, in-work, and post-work phases.
[0096] When the creation unit 195 detects a user tap on an inputtable area, it displays a report form for entering report content into that area. Specifically, for example, when the user taps the first editing area 1416 located below "Before Work," the creation unit 195 displays a report form for entering report content into the first editing area 1416.
[0097] Figure 13(b) is a schematic diagram representing a report form for inputting report content into designated areas. In the example shown in Figure 13(b), the creation unit 195 displays a first editing area 14161 and a second editing area 14162 on the display 141. The first editing area 14161 is an area for attaching images. The second editing area 14162 is an area for inputting text information obtained through diagnosis.
[0098] In step S22, the terminal device 10 attaches an image to the creation screen based on instructions from the user. Specifically, the creation unit 195 attaches, for example, an image selected by the user to a predetermined area of the creation screen.
[0099] In Figure 13(b), when attaching an image to the first editing area 14161, the user taps the first editing area 14161. When the first editing area 14161 is tapped by the user, the creation unit 195 displays, for example, a selection window (not shown) for selecting the image to attach to the first editing area 14161. The user selects the image to attach to the first editing area 14161 from the list of images displayed in the selection window. The creation unit 195 attaches the image selected by the user to the first editing area 14161.
[0100] In step S23, the terminal device 10 reads information about the tags attached to the attached image. Specifically, for example, when an image is attached, the creation unit 195 refers to the image information 184 and obtains information about the tags associated with the attached image.
[0101] In step S24, the terminal device 10 outputs the text related to the read tag to the creation screen. Specifically, for example, the creation unit 195 displays the text related to the read tag on the creation screen so that the user can edit it.
[0102] Figure 13(c) is a schematic diagram showing an example of inputting information into the report form. In the example shown in Figure 13(c), an image is attached to the first editing area 14161. The text of the tag attached to the image is displayed in the second editing area 14162. The text displayed in the second editing area 14162 can be edited by the user by tapping the second editing area 14162. Once the user has finished entering the report content, they press button 14163.
[0103] Figure 13(d) is a schematic diagram showing an example of how the generated report is displayed. In the example shown in Figure 13(d), reports for before and after the work have been entered.
[0104] As described above, in the above embodiment, the shooting control unit 194 extracts candidate tags from a plurality of tags, any of which contain characters related to the object to be photographed, based on information about the shooting. The presentation control unit 196 presents the extracted candidate tags in a selectable format. The shooting control unit 194 accepts a selection from the user for the presented candidate tags. The shooting control unit 194 photographs the object to be photographed. The shooting control unit 194 stores the photographed image, associating it with the tags specified by the user. As a result, when taking an image, tags that are highly likely to be assigned by the user are extracted as candidate tags. Therefore, the terminal device 10 can reduce the effort required of the user to search for the desired tag from a plurality of tags when storing an image.
[0105] Therefore, according to the terminal device 10 of this embodiment, the burden on the operator when managing images can be further reduced.
[0106] Furthermore, in the above embodiment, the shooting control unit 194 extracts tags associated with groups that manage images related to the subject being photographed as candidate tags. As a result, tags suitable for the management group are automatically extracted, and candidate tags are effectively extracted when storing images.
[0107] Furthermore, in the above embodiment, the shooting control unit 194 extracts tags associated with attributes included in the user's personal information as candidate tags. As a result, tags related to the user are automatically extracted, and candidate tags are effectively extracted when storing images.
[0108] Furthermore, in the above embodiment, the shooting control unit 194 extracts tags associated with the object being photographed as candidate tags. As a result, tags related to the object being photographed are automatically extracted, so when storing the image, candidate tags are effectively extracted.
[0109] Furthermore, in the above embodiment, the shooting control unit 194 identifies the object to be photographed by image analysis and extracts tags associated with the identified object to be photographed as candidate tags. As a result, tags related to the object being photographed are automatically extracted, so when storing the image, candidate tags are extracted effectively.
[0110] Furthermore, in the above embodiment, the shooting control unit 194 identifies the object to be photographed based on a code associated with the object, and extracts tags associated with the identified object to be photographed as candidate tags. As a result, tags related to the object being photographed are automatically extracted, so when storing the image, candidate tags are extracted effectively.
[0111] Furthermore, in the above embodiment, the shooting control unit 194 creates tags based on user information. The shooting control unit 194 extracts candidate tags from multiple tags and the created tags. This makes it possible to create tags that are appropriate for the current situation, thereby improving the accuracy of extracting the desired tags.
[0112] Furthermore, in the above embodiment, the presentation control unit 196 presents candidate tags in a tree structure. This allows the extracted candidate tags to be displayed in an efficient and user-friendly manner, reducing the effort required to select a desired tag from the candidate tags.
[0113] Furthermore, in the above embodiment, if no user specifies otherwise, the shooting control unit 194 identifies the target to be photographed and associates the tag associated with the identified target with the image. As a result, even if the user has not selected a tag, the relevant tag will be attached to the image.
[0114] Furthermore, in the above embodiment, the shooting control unit 194 notifies the user that a tag has been associated with the image based on the identified shooting target. This allows the user to automatically recognize that an image has been tagged. Therefore, it becomes possible to pay attention to images that have been automatically tagged.
[0115] Furthermore, in the above embodiment, when an image to which a tag is associated is specified, the creation unit 195 displays the text related to the tag associated with the image in an editable format. As a result, when creating a report, simply selecting an image automatically inputs the words necessary to create the report. Also, since the report is created based on tags, the amount of text input is reduced. Therefore, reports can be created efficiently even when using a smartphone. Because reports can be created on a smartphone, there is no need to return to the base after the diagnosis and create the report on a terminal at the base. This significantly reduces the burden on the user.
[0116] Furthermore, in the above embodiment, the shooting control unit 194 takes a photograph of the object to be photographed. The shooting control unit 194 extracts candidate tags from a plurality of tags, at least one of which contains characters related to the object to be photographed, based on information about the photograph. The shooting control unit 194 presents the extracted candidate tags in a selectable format and accepts selection from the user. The shooting control unit 194 stores the photographed image, associating it with the tags specified by the user. As a result, when taking a photograph, tags that are highly likely to be assigned by the user are extracted as candidate tags. Therefore, when the terminal device 10 stores an image, it is possible to reduce the effort required of the user to search for the desired tag from a plurality of tags.
[0117] <Variation> In the above embodiment, after selecting a tag, a storage instruction for the image being captured is entered. The image captured when the storage instruction is entered is then stored in association with the selected tag. However, selecting a tag is not limited to before entering the storage instruction. The shooting control unit 194 may allow the user to select a tag after the storage instruction for the image being captured has been entered.
[0118] Figure 14 is a flowchart illustrating examples of other operations of the terminal device 10 when storing image data. When the user needs to store an image of a specific object while performing a predetermined diagnosis, the user operates the terminal device 10 to photograph the object.
[0119] In step S31, the terminal device 10 receives an instruction to store the captured content. The user presses, for example, the button 1412 shown on the left in Figure 10. The shooting control unit 194 determines that the user has input an instruction to store the image when the button 1412 is pressed.
[0120] In step S32, the terminal device 10 acquires diagnostic information. Specifically, for example, when an instruction to store an image is input, the shooting control unit 194 reads the user's affiliation stored in the user information 181 as a related attribute. The shooting control unit 194 also acquires the user's schedule and, based on the acquired schedule, acquires the currently running appointments. The shooting control unit 194 acquires the equipment or facilities included in the acquired appointments as related equipment. The shooting control unit 194 also acquires the equipment or facilities where the user is currently located as related equipment, based on the current location acquired by the location information sensor 150. The shooting control unit 194 also recognizes the device to be photographed and identifies the device. The shooting control unit 194 acquires the identified device as related equipment. The diagnostic information acquired by the shooting control unit 194 may be only a part of the above, or other different information may be added.
[0121] In step S33, the terminal device 10 extracts candidate tags. Specifically, for example, the imaging control unit 194 extracts candidate tags from a plurality of tags stored in the tag information 182 based on the configured management group, acquired diagnostic information, or a combination thereof.
[0122] In step S34, the terminal device 10 presents the extracted candidate tags to the user. Specifically, for example, the presentation control unit 196 displays the extracted candidate tags on the display 141 so that the user can select them. The user selects a desired candidate tag from the multiple candidate tags displayed on the display 141.
[0123] In step S35, the terminal device 10 accepts the user's selection of candidate tags. Specifically, for example, the shooting control unit 194 accepts the user's selection instruction for one of several candidate tags displayed on the display 141. Once the user has selected a tag, they complete the selection by pressing a predetermined icon.
[0124] In step S36, the terminal device 10 stores the image data. Specifically, once the tag selection is complete, the presentation control unit 196 stores the image that was captured when the button 1412 was pressed in the image information 184, associating it with the selected tag.
[0125] In step S33, the shooting control unit 194 may create tags based on, for example, the user's current state. Specifically, when the user instructs the shooting control unit 194 to extract candidate tags, it may create tags based on, for example, wording included in the user's schedule. The shooting control unit 194 may also create tags based on, for example, wording related to the user's current location. The shooting control unit 194 extracts candidate tags from the tags stored in the tag information 182 and the tags it has created. The shooting control unit 194 may also create tags in response to the user's login.
[0126] The shooting control unit 194 may, for example, if no tag is specified in step S35, acquire a tag from candidate tags based on various information and associate the acquired tag with the stored image data. Specifically, for example, the shooting control unit 194 identifies the target to be photographed by image analysis or based on a predetermined code attached to the target. The shooting control unit 194 acquires a tag that the identified target to be photographed is stored as an associated device. The shooting control unit 194 also grasps the user's current state, for example, based on the current location or schedule. The shooting control unit 194 acquires a tag that the grasped state is stored as an associated device.
[0127] Furthermore, in the above embodiment, the process described in Figures 9, 11, and 14 was explained using the terminal device 10 as an example. However, the process described in Figures 9, 11, and 14 is not limited to the terminal device 10. Some or all of the process described in Figures 9, 11, and 14 may be performed by the server 20. In other words, the user gives instructions to the server 20 via the terminal device 10, and the server 20, upon receiving the instructions, executes the process. At this time, user information 181, tag information 182, related information 183, image information 184, and report information 185 stored in the terminal device 10 are stored in the server 20 as needed.
[0128] Furthermore, the above embodiment describes a case in which the imaging control unit 194 extracts candidate tags from multiple tags stored in the tag information 182 based on a set management group, acquired diagnostic information, or a combination thereof. However, the information referenced when extracting candidate tags is not limited to the management group and diagnostic information. For example, any information related to imaging, such as information related to the examination or evaluation, may be used.
[0129] Furthermore, the above embodiment described a case in which, for example, related attributes, related devices, related equipment, and management groups are associated with the tags. However, these pieces of information do not necessarily have to be associated with the tags. A trained model may be used that is trained to output tags that are highly likely to be assigned based on the information at the time of shooting and the image being captured. The shooting control unit 194, for example, inputs multiple stored tags, information at the time of shooting, and the image being captured into the trained model and outputs multiple tags that are highly likely to be assigned to the image as candidate tags.
[0130] Furthermore, in the above embodiment, when creating a report, a case was described in which text related to the tags associated with the image is input when an image is attached. However, the operation when creating a report is not limited to this. A trained model that has been trained to output a text sentence representing the content of the report based on the image stored during the diagnosis and the tags associated with the image may be used. For example, the creation unit 195 inputs the image attached by the user and the tags associated with the image into the trained model and causes it to output a text sentence representing the content of the report. The trained model may be a generative AI such as a large-scale language model. In this case, for example, the creation unit 195 inputs the image attached by the user, the tags associated with the image, and a prompt to output the report into the generative AI and causes it to output a text sentence representing the content of the report.
[0131] <4. Basic Hardware Configuration of a Computer> Figure 15 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 91, main memory 92, auxiliary memory 93, and a communication interface 99. These are electrically connected to each other by a bus.
[0132] The processor 91 is hardware for executing the instruction set written in the program. The processor 91 consists of an arithmetic unit, registers, peripheral circuits, etc.
[0133] Main memory 92 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).
[0134] Auxiliary storage device 93 is a storage device for storing data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc.
[0135] A communication IF99 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards. A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.
[0136] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.
[0137] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 shown in Figure 15 will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.
[0138] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.
[0139] The control unit is realized when the processor 91 reads various programs stored in the auxiliary storage device 93, loads them into the main memory device 92, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.
[0140] The memory unit is implemented by a main memory 92 and an auxiliary memory 93. The memory unit stores data, various programs, and various databases. The processor 91 can also reserve memory areas corresponding to the memory unit in the main memory 92 or the auxiliary memory 93 according to the program. The control unit can also cause the processor 91 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.
[0141] A database, specifically a relational database, is used to manage and link data sets called tables, which are structurally defined by rows and columns. In a database, tables are called tables, the columns of a table are called columns, and the rows of a table are called records. In a relational database, relationships can be established and linked between tables. Typically, each table has a key column to uniquely identify records, but setting a key on a column is not mandatory. The control unit can instruct the processor 91 to add, delete, or update records in specific tables stored in the memory unit according to various programs.
[0142] The communication unit is implemented by the communication IF99. The communication unit implements the function of communicating with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 91 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.
[0143] The functions realized by the components described herein may be implemented in a circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor includes transistors and other circuits and is considered a circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in memory. In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein. If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.
[0144] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.
[0145] <Note> The details described in each of the above embodiments are noted below. (Note 1) A program for execution on a computer having a processor and memory, the program causing the processor to perform the following steps: extract candidate tags from a plurality of tags, any of which contain characters relating to a subject to be photographed, based on information about the subject to be photographed; present the extracted candidate tags in a selectable format; accept a user's specification for the presented candidate tags; photograph the subject to be photographed; and store the photographed image in association with the tags specified by the user. (Note 2) The program described in Appendix 1 extracts tags associated with groups that manage images related to the subject being photographed as candidate tags during the extraction step. (Note 3) The program described in (Appendix 1) or (Appendix 2) extracts tags associated with attributes included in the user's personal information as candidate tags during the extraction step. (Note 4) A program as described in any of (Appendix 1) to (Appendix 3), which extracts tags associated with the subject being photographed as candidate tags in the extraction step. (Note 5) The program described in Appendix 4 identifies the target object by image analysis during the extraction step and extracts tags associated with the identified target object as candidate tags. (Note 6) The program described in Appendix 4 identifies the subject based on the code associated with the subject during the extraction step, and extracts tags associated with the identified subject as candidate tags. (Note 7) A program as described in any of (Appendix 1) to (Appendix 6), which causes a processor to perform the step of creating tags based on user information, and in the extraction step, extracts candidate tags from multiple tags and the created tags. (Note 8) A program that presents candidate tags in a tree structure in the step of presentation, as described in any of (Appendix 1) to (Appendix 7). (Note 9) In the memory step, if the user does not specify otherwise, the program identifies the subject to be photographed and associates the tag associated with the identified subject to be photographed with the image, as described in any of the (Appendix 1) to (Appendix 8) below. (Note 10) The program described in Appendix 9 notifies the user that, in the memory step, a tag has been associated with an image based on the identified subject being photographed. (Note 11) A program as described in any of (Appendix 1) to (Appendix 10), which, when an image to which a tag is associated is specified, causes the processor to perform the step of displaying the text related to the tag associated with the image in an editable format. (Note 12) A program for execution on a computer having a processor and memory, the program causing the processor to perform the following steps: taking a photograph of a subject to be photographed; extracting candidate tags from a plurality of tags, any of which contain characters relating to the subject to be photographed, based on information about the photograph; presenting the extracted candidate tags in a selectable format and accepting selection from the user; and storing the photographed image in association with the tags specified by the user. (Note 13) A method to be performed on a computer comprising a processor and memory, wherein the processor performs all steps of any of the inventions described in (Appendix 1) to (Appendix 12). (Note 14) An information processing apparatus comprising a processor and memory, wherein the processor performs all the steps of any of the inventions described in (Appendix 1) to (Appendix 12). (Note 15) A system comprising means for performing all the steps of any of the inventions described in (Appendix 1) to (Appendix 12). [Explanation of symbols]
[0146] 1... System 10…Terminal device 120... Communications Department 13…Input device 131…Touch-sensitive devices 14…Output device 15…Memory 16…Storage 19… Processor 20... Server 22...Communication IF 23…Input / Output Interface 25…Memory 2 hours… storage 29… Processor
Claims
1. A program to be executed by a computer having a processor and memory, wherein the program is to be executed by the processor, The process involves extracting candidate tags from multiple tags, each containing text related to the subject being photographed, based on information about the photograph. The steps include presenting the extracted candidate tags in a selectable format, A step of receiving user specifications for the aforementioned candidate tags presented, The steps of photographing the subject, The steps include storing the captured image while associating it with a tag specified by the user, and A program that executes the command.
2. The program according to claim 1, wherein in the extraction step, tags associated with a group that manages images related to the subject being photographed are extracted as candidate tags.
3. The program according to claim 1, wherein in the extraction step, tags associated with attributes included in the user's personal information are extracted as candidate tags.
4. The program according to claim 1, wherein in the extraction step, tags associated with the subject being photographed are extracted as candidate tags.
5. The program according to claim 4, wherein in the extraction step, the program identifies the target to be photographed by image analysis and extracts tags associated with the identified target to be photographed as candidate tags.
6. The program according to claim 4, wherein in the extraction step, the program identifies a subject based on a code associated with the subject, and extracts tags associated with the identified subject as candidate tags.
7. The processor is instructed to perform the step of creating a tag based on information relating to the user. The program according to claim 1, wherein in the extraction step, candidate tags are extracted from the plurality of tags and the created tags.
8. The program according to claim 1, wherein in the step described above, the candidate tags are presented in a tree structure.
9. The program according to claim 1, wherein, in the memory step, if no specification is made by the user, the program identifies the subject to be photographed and associates the tag associated with the identified subject to be photographed with the image.
10. The program according to claim 9, wherein in the memory step, the program notifies the user that the tag has been associated with the image based on the identified subject being photographed.
11. The program according to claim 1, which, when an image to which the tag is associated is specified, causes the processor to perform the step of displaying the text related to the tag associated with the image in an editable format.
12. A program to be executed by a computer having a processor and memory, wherein the program is to be executed by the processor, The steps of photographing the subject, The process involves extracting candidate tags from multiple tags, each containing text related to the subject being photographed, based on information about the photograph. The steps include presenting the extracted candidate tags in a selectable format and accepting selections from the user, The steps include storing the captured image while associating it with a tag specified by the user, and A program that executes the command.
13. A method to be performed on a computer comprising a processor and memory, wherein the processor performs all the steps of any of the inventions according to claims 1 to 12.
14. An information processing apparatus comprising a processor and memory, wherein the processor performs all the steps of any of the inventions according to claims 1 to 12.
15. A system comprising means for performing all the steps of any of the inventions according to claims 1 to 12.