Information presentation system, unit support information generation device, information presentation method, unit support information generation method, program, and non-transitory computer-readable recording medium

Abstract

The present invention provides an information presentation technique for presenting unit support information to an operator in an easy-to-understand manner even when a provider is absent, and for suppressing the reproduction load when changing a workflow.　An information presentation system 100 performs the following: on the basis of an identifier, acquiring a plurality of pieces of unit support information including at least a still image and short-term audio, and workflow support information defining a presentation order or a transition condition of the plurality of pieces of unit support information; displaying the still image included in the selected unit support information and outputting the short-term audio; controlling the transition between the pieces of unit support information on the basis of the workflow support information; and maintaining display of the still image even after completion of audio output. The unit support information and the workflow support information are stored as separate data that can refer to each other, and the workflow support information can be independently updated in a mode that does not require reproduction of the still image and the short-term audio included in the unit support information.

Description

Information presentation system, unit support information generation device, information presentation method, unit support information generation method, program, and non-transitory computer-readable recording medium

[0001] The present invention relates to an information presentation technology for presenting work procedures, confirmation items, or precautions to workers, and relates to an information presentation system, unit support information generation device, information presentation method, unit support information generation method, program, and non-transitory computer-readable recording medium that present information to workers using still images and short-duration audio recorded on site by experienced persons in spatial tasks such as maintenance, inspection, assembly, construction, transportation, facility management, article management, building, nursing care, agriculture, and disaster prevention.

[0002] As used herein, "spatial task" refers to a task that is performed by visually recognizing a target object, facility, member, location, or surrounding environment in real space, and accompanied by body movement, line-of-sight movement, or manual operation.

[0003] Conventionally, paper manuals, electronic manuals, continuous moving images, audio guidance, augmented reality (AR) display and the like have been used as work support for workers.

[0004] However, paper manuals and list-type electronic manuals impose a heavy burden on workers to search for required sections by themselves, and require frequent back-and-forth between actual object confirmation and information confirmation. In addition, with continuous moving images, workers need to follow the time progression on the presentation side, and missed content and rewind operations are prone to occur. Furthermore, AR displays are susceptible to influences such as alignment, superimposition accuracy, wearability of display devices, ambient light, and occlusion.

[0005] In spatial tasks, workers need to understand the next task to be performed while confirming the shape, position, orientation, surrounding conditions, and safety conditions of the actual object. If the presentation of visual and auditory information is inappropriate, cognitive resources will be spent on following the information rather than understanding the task itself. Cognitive resources are the brain's processing capacity consumed when performing multiple tasks simultaneously, and this processing capacity is finite. Cognitive resources may include working memory, which is involved in the temporary holding and processing of information. When the amount of information processed simultaneously increases, processing delays or decreased attention allocation occur. In particular, working memory has an upper limit to the amount of information it can hold, and if information exceeding this limit is input, the accuracy of information processing decreases, and work errors may occur.

[0006] Regarding the limitations of short-term memory and working memory, the following works are known: Miller, G. A., "The Magical Number Seven, Plus or Minus Two" (1956), Baddley, A., "Working Memory" (1992), and Cowan, N., "The Magical Number 4 in Short-Term Memory" (2001). Furthermore, Mayer, R. E., "Multimedia Learning" (3rd ed., 2021) also indicates that presenting information in segmented form and appropriately combining visual and auditory channels contributes to supporting comprehension. The inventors of this invention have observed that, even when presenting information during spatial tasks, combining short, segmented audio clips with still images that can be grasped at a glance is more effective than continuous, lengthy explanations in allowing workers to retain information while physically examining the objects.

[0007] The work know-how of experienced or knowledgeable individuals (skilled workers, etc.) is not adequately structured in paper procedure manuals or sequential videos, and is easily lost due to its individual nature. Traditional work support methods require face-to-face instruction by experienced individuals, resulting in labor shortages and significant training costs.

[0008] Furthermore, since the target equipment, surrounding environment, work sequence, language used, and worker skill level differ from site to site, a flexible and replaceable information structure is desirable.

[0009] For example, if the technology described in Patent No. 7219932 is centered on video or video-like process presentation, the presentation medium itself progresses over time, making it burdensome for the worker to keep up with the progress of the presentation. In contrast, the present invention adopts a two-layer structure in which each work unit is made independent as unit support information consisting of a still image and a short audio clip, and the transitions between these units are controlled by workflow support information. As a result, the worker can refer to the still image and confirm the actual product even after the audio clip has finished.

[0010] Furthermore, if the technology described in Patent No. 7688867 utilizes AR display, then the core of that technology is superimposition or alignment onto the target object. On the other hand, the present invention does not require AR superimposition and employs a configuration that uses still images and short audio clips as basic units to perform transition control based on image comparison, location information, spatial range information, or external data, making it easier to implement even with simpler terminals.

[0011] Patent No. 7219932, Patent No. 7688867, Patent No. 7651232

[0012] Miller, G. A. , “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information”, Psychological Review, 1956. Baddeley, A. , “Working Memory”, Science, 1992. Cowan, N. , “The magical number 4 in short-term memory: A reconciliation of mental storage capacity”, Behavioral and Brain Sciences, 2001. Mayer, R. E. , “Multimedia Learning”, 3rd ed. , Cambridge University Press, 2021.

[0013] The present invention has been made in view of the above problems, and aims to provide an information presentation technology that facilitates the generation, updating, and modification of support information for each site by presenting still images and short audio clips corresponding to each work unit in a spatial task in accordance with the worker's confirmation timing, maintaining the display of still images even after audio output to ensure that the worker has time to confirm the actual object, and enabling the separate management of the presented content (unit support information) and the presentation order or transition conditions (workflow support information).

[0014] To solve the above problems, an information presentation system according to one aspect of the present invention comprises: an identifier receiving means for receiving an identifier; a unit support information acquisition means for acquiring a plurality of unit support information, including at least a still image and audio of a predetermined length or less, based on the identifier; a workflow support information acquisition means for acquiring workflow support information that defines the presentation order, branching conditions, transition conditions, and priority of the plurality of unit support information; a presentation control means for displaying the still image included in the selected unit support information and outputting the audio; and a transition control means for controlling the transition between the plurality of unit support information based on the workflow support information, wherein the unit support information and the workflow support information are generated, updated, and managed independently of each other, and the system has a two-layer structure.

[0015] In the present invention, the unit support information and the workflow support information are stored as separate data that can refer to each other, and changes to the workflow support information can be updated independently in a manner that does not require the regeneration of still images and audio included in the unit support information.

[0016] In the first mode, the presentation control means maintains the display of the still image even after the output of the sound has ended, accepts the next operation input from the operator, and presents the next unit support information in accordance with the next operation input. In the second mode, the presentation control means presents the next unit support information without requiring the next operation input when a transition condition including location information, the result of comparison of captured images, or external state information is met.

[0017] The unit support information preferably includes at least one still image and at least one short audio clip. It may also include, as necessary, provider information, reference images, notes, language information, location information, spatial range information, identification information, and other auxiliary information.

[0018] In this specification, "short audio (audio of a predetermined length or less)" refers to audio divided into segments of a length that allows the worker to easily retain the content while checking the actual object, preferably about 15 seconds or less, and more preferably a few seconds to about 15 seconds. The length of about 15 seconds is an implementation guideline that is consistent with the constraints of short-term memory and working memory shown in non-patent literature, while also ensuring compatibility between on-site listening and physical object verification. If necessary, the explanatory content may be divided into multiple unit support information segments.

[0019] With this configuration, workers do not need to constantly follow the progress of a continuous video; they can grasp the reference object through still images and receive the minimum necessary explanations through short audio clips.

[0020] Furthermore, by maintaining the display of a still image even after the short audio playback has finished, workers can proceed with their work while confirming the actual object using the explanatory content as a guide. This ensures that workers have both temporal and spatial leeway, allowing them to refer back and forth between the actual object, the surrounding environment, and the displayed content as needed, without being constrained by the time progression of the display medium.

[0021] Furthermore, by introducing transition control that utilizes location information or spatial range information, the displayed content can be switched according to the worker's movement or the target location, enabling more natural work support.

[0022] This figure shows an example of the overall configuration of an information presentation system according to one embodiment of the present invention. This figure shows an example of the functional block configuration of a presentation terminal. This figure shows an example of the functional block configuration of a management device. This figure shows an example of the functional block configuration of a unit support information generation device. This figure shows an example of the data structure of unit support information. This figure shows an example of the data structure of workflow support information. This figure shows an example of a presentation screen including still images, provider information, and reference images. This figure shows an example of the basic flow of information presentation processing. This figure shows an example of the flow of unit support information generation processing. This figure shows the types of transition methods. This figure shows the processing flow during offline operation. This figure shows an example of the configuration of a management screen. This figure shows an example of the display of provider information. This figure shows a history management and analysis screen. This figure shows the presentation of identifier candidates through location information linkage. This figure shows the flow of movement-synchronous unit support information generation.

[0023] Embodiments of the present invention will now be described with reference to the drawings. The information presentation system 100 shown in Figure 1 includes a presentation terminal 110, a management device 120, and, if necessary, a unit support information generation device 130. The presentation terminal 110 may be a terminal carried or worn by an operator, or it may be a display terminal installed near the equipment. Examples of terminals carried and used include smartphones and tablet terminals, while examples of terminals worn and used include smart glasses. The management device 120 may be built on a cloud environment, in which case the presentation terminal 110 connects to the management device 120 via the Internet or a network using HTTP, HTTPS, or API communication.

[0024] The information presentation system 100 shown in Figure 1 can provide an alternative to face-to-face instruction by asynchronously presenting audio and still images recorded on-site by a provider with experience or knowledge of the work object or work process to the worker at a time different from the time the provider recorded them. Note that the provider is not limited to a person with experience or knowledge (such as a skilled worker); the worker themselves or the learner may also be the recorder.

[0025] As shown in Figure 1, the presentation terminal 110 may include an identifier receiving unit 111, a unit support information acquisition unit 112, a workflow support information acquisition unit 113, a presentation control unit 114, a transition control unit 115, a display unit 116, an audio output unit 117, a shooting unit 118, and a local storage unit 119. The management device 120 may include a communication unit 121, a storage unit 122, a control unit 123, a translation processing unit 124, a history management unit 125, a location linkage unit 126, and a management interface unit 127. Figure 2 shows an example of the functional block configuration of the presentation terminal 110. As shown in Figure 2, each part of the presentation terminal 110 cooperates with each other to realize a series of processes from identifier reception to information acquisition, presentation control, transition control, and local storage. Figure 3 shows an example of the functional block configuration of the management device 120. As shown in Figure 3, each part of the management device 120 is responsible for the storage and provision of unit support information and workflow support information, as well as the functions of translation, history management, location linkage, and management interface. Figure 4 shows an example of the functional block configuration of the unit support information generation device 130. As shown in Figure 4, the unit support information generation device 130 may include an image acquisition unit 131, an audio acquisition unit 132, an association unit 133, a position / time acquisition unit 134, a unit generation unit 135, a workflow candidate generation unit 136 as needed, and an output control unit 137 as an optional configuration. The output control unit 137 outputs an audio inquiry to the provider, and triggers the audio acquisition unit 132 to start audio acquisition in response to the inquiry.

[0026] The management device 120 may optionally include an API provision unit 128. The API provision unit 128 provides unit support information, workflow support information, or related metadata to an external device or presentation terminal 110 according to the identifier, user identification information, target equipment identification information, language information, work mode information, and other requested information.

[0027] The API provider 128 may accept communications using, for example, HTTP, HTTPS, gRPC, or WebSocket, and provide information in JSON, XML, or other predetermined formats. In addition to requests for information acquisition, the API provider 128 may also accept requests for registration, updating, or deletion of unit support information, workflow support information, identifier correspondence information, or auxiliary information.

[0028] Unit support information includes at least still images and short audio clips. Still images may be images showing the work object, components, location, posture, completed state, points to check, or points to pay attention to, and short audio clips may be audio clips indicating work instructions, precautions, judgment criteria, or points to check that correspond to the still images. In other words, unit support information targeting points to check or points to check can be said to correspond to the check points in the work procedure. Here, "unit support information" refers to information for each work unit that includes at least still images and short audio clips.

[0029] Unit support information may include multiple still images for a single audio. In this specification, "multiple still images" means multiple still images included in a single unit support information, and refers to a combination of images of the same work object taken from different angles or at different magnifications, images for comparing the state before and after work, or images that show the points to be checked in stages.

[0030] Workflow support information is information that defines the sequential relationships, branching conditions, transition conditions, priorities, termination conditions, and, if necessary, return conditions of multiple unit support information. For example, each unit support information may be associated with and stored the next candidate unit ID, conditional expression, priority, skipability, re-presentation capability, etc. Here, "workflow support information" refers to information that defines the presentation order, branching conditions, transition conditions, and priorities between multiple unit support information.

[0031] The identifier receiving unit 111 may receive identifiers by means of barcodes, QR codes (registered trademarks), NFC tags, RFID tags, equipment numbers, location IDs, work order IDs, voice input, string input, or other methods.

[0032] In this specification, "identifier" refers to information used to identify unit support information or workflow support information, and may consist of one or more pieces of information such as a target equipment ID, work target ID, work order ID, location ID, process ID, user ID, language code, etc. The identifier receiving unit 111 may receive a single code, or it may receive multiple elements and generate an identifier by combining them.

[0033] The unit support information acquisition unit 112 may acquire multiple unit support information corresponding to the identifier from the management device 120, local storage unit 119, or an external system. The workflow support information acquisition unit 113 may acquire workflow support information corresponding to the identifier or the acquired group of units. This realizes a two-tiered structure in which multiple independent unit support information and workflow support information relating them are acquired based on the identifier.

[0034] The local memory unit 119 may store frequently used unit support information, workflow support information, reference images, translated audio, or history information. This makes it easier to continue presentation even in environments with unstable communication.

[0035] The translation processing unit 124 may generate or manage text, audio, or subtitle information in multiple languages ​​in response to the text or audio included in the unit support information. The presentation control unit 114 may switch the display content and audio according to user settings or language information.

[0036] The display control unit 114 displays a still image included in the selected unit support information on the display unit 116 and outputs a short audio message from the audio output unit 117. If necessary, text, provider information, reference images, warning icons, progress status, or next candidate information may be displayed near the still image.

[0037] The display control unit 114, when a single unit support information includes multiple still images, can switch the currently displayed still image to another still image via a swipe operation, voice command, or button operation by the operator, both during and after audio playback. This switching operation can be performed without interrupting audio playback, allowing the operator to refer to multiple still images at any time while performing tasks and movements while listening to the audio.

[0038] As a display method for multiple still images, at least one of the following can be adopted as a static image: a sequential display method that switches one image at a time in response to the operator's operation, a thumbnail method that displays multiple still images as reduced thumbnails in a list, and a slideshow method that automatically switches sequentially at a preset interval. By using static images, unlike videos that progress over time, the operator can proceed with their work while referring to the displayed content at their own pace of review. The display method can be set for each unit support information and can be selected according to the nature of the work or the characteristics of the object to be reviewed.

[0039] The storage unit 122 may store the display order, magnification information, shooting direction information, or explanatory label associated with each of the multiple still images. The display control unit 114 can refer to this additional information obtained from the storage unit 122 to control the display order of the multiple still images and automatically adjust the magnification.

[0040] Provider information may include information indicating the creator, verifier, department, creation date and time, update date and time or approval status of the unit support information, as well as the provider's skill level, qualifications, or relationship to the worker (same site, same job type, designated person, recommender, etc.). By referring to the provider information, workers can use the information while recognizing which experienced person it is based on. Reference images may be images showing the pre-work state, post-work state, comparison state, or state requiring attention. Displaying these makes it easier for workers to make judgments while confirming the reliability of the presented information.

[0041] Short audio clips complement the content of still images by concisely explaining the object's name, areas to check, tools, direction, torque, precautions, and completion conditions. The audio may be played only once, or it may be playable, stopped, and restarted as needed by the operator. The length of the audio clips may be set to be less than or equal to a predetermined time; for example, setting them to approximately 15 seconds or less makes it easier for the operator to retain the information while checking the actual object. If the amount of explanation is extensive, the audio clips can be divided into multiple unit support information sections, with each section assigned to a different audio clip, thus keeping each clip within the predetermined time limit.

[0042] The presentation control unit 114 may maintain the display of the still image even after the reproduction of the short-time audio ends, at least until the next transition or an operation by the worker occurs. This makes it easier for the worker to confirm the actual object and continue the work using the audio content as a clue.

[0043] The transition control unit 115 controls the transition from the currently presented unit support information to the unit support information to be presented next, based on the workflow support information. The transition may be a manual transition, an automatic transition, or a conditional transition.

[0044] As an example of the automatic transition, the transition may be determined based on the comparison result between the image acquired by the imaging unit 118 and the reference image or still image corresponding to the current or next candidate unit. As the comparison processing, pixel difference, feature amount matching, matching degree determination by a trained model, or other image comparison methods may be used. For example, the automatic transition may be executed when the matching degree is equal to or higher than a predetermined threshold, or when the difference amount is equal to or lower than a predetermined threshold. The threshold may be settable for each piece of unit support information, and can be adjusted according to the strictness of the work and the characteristics of the confirmation target.

[0045] In this system, the determination of automatic transitions is made based on various transition conditions such as the aforementioned location information, image comparison, equipment operating status, or sensor values. In the first mode (manual transition priority mode), even if the transition conditions are met, the presentation control unit 114 maintains a standby state for the worker to compare the presented information with the actual object and make their own judgment, and only executes the transition to the next unit after detecting that an active operational intervention has been input by the worker. In the second mode (automatic transition priority mode), when transition conditions such as the satisfaction of location information are met, the presentation control unit 114 automatically transitions to the next unit without waiting for operational intervention by the worker, and the presentation is synchronized with the worker's movement pace. In the first mode, the transition control unit 115 determines the destination unit based on workflow support information when the presentation control unit 114 receives manual operational input from the worker, and notifies the presentation control unit 114. In the second mode, the transition control unit 115 monitors transition conditions including location information, comparison results of captured images, sensor values, or external state information, and when it detects that the conditions are met, it determines the destination unit based on workflow support information and notifies the presentation control unit 114.

[0046] Here, "spatial range information" refers to spatial condition information, including a presentation start range, a presentation continuation range, or an exit determination range, associated with unit support information. Spatial range information may include at least one of the presentation start range, presentation continuation range, and exit determination range. The presentation start range is the range for initiating the presentation of unit support information in response to the worker's entry. The presentation continuation range is the range for continuing the presentation of the unit support information after the presentation has started, and the presentation is maintained as long as the worker remains within this range. The exit determination range is the range for updating the status to "presented," excluding it from presentation candidates, or setting conditions for re-presentation in response to the worker moving outside of that range. These ranges may be the same as each other, or they may be set to different sizes or shapes depending on the application. For example, by setting the presentation start range to be the widest, the presentation continuation range to be narrower than the presentation start range, and the exit determination range to be the narrowest, the sensitivity of presentation control to worker movement can be flexibly adjusted.

[0047] As another example, transition based on position information may be performed. Starting presentation, continuing presentation, transitioning, skipping, or re-presenting may be determined depending on whether position information obtained from a presentation terminal 110 or an external sensor is included in, close to, or out of the spatial range information associated with the unit support information.

[0048] By collating the position information or sensing results with the spatial range information corresponding to each piece of unit support information, processes including presentation start, candidate extraction, priority control, range exit determination, presented status update, and re-presentation can be performed. This realizes movement-synchronized presentation of unit support information.

[0049] For example, when a plurality of candidate ranges overlap, unit support information with a higher priority is preferentially presented, and when the user moves outside the exit determination range, controls may be performed such as treating the relevant unit support information as already presented, temporarily excluding it from candidates, or updating it to a state where it can be re-presented upon re-approach. The priority may be set by an administrator as part of the workflow support information, and can be specified as a numerical value or rank according to, for example, the importance of the work, the urgency of confirmation, the hazard level of equipment, or the process sequence.

[0050] In a case where a plurality of pieces of unit support information are presented in an ordered manner, the workflow support information may designate origin unit support information that serves as the origin of the sequence. Until the operator enters the spatial range information corresponding to the origin unit support information and the presentation of the relevant unit support information is performed, the transition control unit 115 does not start presenting the subsequent unit support information even if the operator enters the spatial range information corresponding to the subsequent unit support information. This prevents the operator from starting presentation from an intermediate piece of unit support information without going through the origin in a scene where a plurality of ordered pieces of unit support information are presented, and maintains the consistency of the presentation sequence. Note that when no origin unit support information is designated, or when there is spatial range information corresponding to another piece of unit support information outside the spatial range information corresponding to the relevant unit support information, the above control is not applied, and the other piece of unit support information may be presented in accordance with normal priority control.

[0051] Regarding the handling of unit support information that has been updated to a presented state, the transition control unit 115 may, when it detects that the worker's position information has moved outside the spatial range information of all unit support information, reset the state of the unit support information that has been updated to a presented state up to that point and return it to an initial state where it can be presented again. Furthermore, if the worker enters the spatial range information of another adjacent unit support information before leaving a certain spatial range information, the current unit support information may be updated to a presented state and the presentation of the adjacent unit support information may be started consecutively. This control makes it possible to continue and switch presentations in a way that smoothly follows the worker's movement, even when the spatial range information of the items to be presented is arranged consecutively or overlapping.

[0052] Furthermore, the transition control unit 115 may exclude and skip unit support information from presentation if its spatial range information is located outside the departure determination range of the currently presented unit support information, among the ordered plurality of unit support information. This is to prevent the unnecessary presentation of unit support information not included in the path when an operator moves through a series of spatial range information in order, thereby reducing redundancy in presentation. The skipped unit support information may be updated to a presented state, and the subsequent re-presentation conditions may be set according to the workflow support information.

[0053] Furthermore, location and image conditions may be used in combination, such as extracting candidate unit support information based on location information and then performing final transition determination by image comparison.

[0054] In addition to the image comparison or location information mentioned above, the transition conditions may also include equipment operating status, sensor values, work completion input, check results, status information acquired from external devices, etc. Furthermore, the worker's gaze direction, body movements, or biometric information (heart rate, fatigue level, etc.) acquired by a display terminal or external sensor may be used as transition conditions. For example, the system may be configured to automatically execute a transition when it is detected that the worker's gaze has been directed towards a predetermined confirmation point for a certain period of time or longer.

[0055] Even when automatic transitions are set, it is preferable to retain manual operation by the operator. This allows the operator to proceed to the next unit, go back, or restart the process at their own discretion, even in the event of environmental changes, sensor errors, or exceptional operations. In other words, these manual operations are positioned as manual fallback operations in relation to automatic transitions. Here, "manual fallback operation" refers to an operation in which, when automatic transitions are set, the operator manually instructs the system to proceed to the next unit, return to the previous unit, re-present, or restart the process based on their own judgment.

[0056] However, regardless of which automated judgment method is used, the final safety confirmation or work execution decision may be made by the worker or manager. The present invention does not replace the worker's judgment, but rather provides information and transition control to support the decision-making process.

[0057] Unit support information can be recorded and generated when a provider with experience or knowledge of the work object or work process explains it in front of the work object. For example, when a provider starts an audio explanation using the so-called "sync-around" method, in which they verbally explain the confirmation details and the basis for their judgments in front of the work object, the audio acquisition unit 132 acquires the audio information, and the image acquisition unit 131 can automatically capture a still image of the work object upon detection of the start of audio input.

[0058] This method of detecting the start of voice input and automatically acquiring still images as a result of that detection is called a "synchronous acquisition method." This method ensures that the provider's voice and the still image of the subject being described are recorded in a time-correspondent manner. Since the still image reflects the provider's perspective at the site, when it is presented to the worker, they can view the subject from the same perspective as the provider.

[0059] When the provider is in front of a specific work object, the output control unit 137 outputs an audio question to the provider. When the provider responds to this question, the audio acquisition unit 132 starts acquiring the audio and the synchronization acquisition means starts capturing the audio, thereby acquiring the provider's response. This response may contain work tips and safety points that the provider unconsciously holds, making it possible to efficiently extract them. Furthermore, since this acquisition is performed in synchronization with the work object, it enables active physical verification, and as a result, it can be recorded as structured knowledge associated with the provider's perspective.

[0060] During playback, the system allows for proactive physical verification at the worker's pace, enabling the worker to perform tasks based on their own judgment while understanding the temporal sequence and spatial situation. Furthermore, if the worker makes a sound after completing the previous task, the system may be configured to analyze the sound using speech recognition processing and transition to the next unit support information upon detection of a predetermined keyword or acknowledgment. Additionally, the system may be configured to detect the worker's gesture operations using the camera or sensors of the display terminal, or to use the detection results as transition operations using location information.

[0061] The unit support information generation device 130 may include an image acquisition unit 131, an audio acquisition unit 132, an association unit 133, a position / time acquisition unit 134, a unit generation unit 135, and, if necessary, a workflow candidate generation unit 136.

[0062] The image acquisition unit 131 may acquire a still image at the timing when the operator or provider starts explaining, when audio recording starts, or at a predetermined shooting operation timing. This makes it easy to generate a visual reference image that corresponds to the audio explanation.

[0063] The voice acquisition unit 132 may record explanatory voice from the worker or provider. The length of the recorded voice can be kept below a predetermined time by automatically stopping the recording after a predetermined time has elapsed from the start of recording, or by extracting a portion within a predetermined time after recording. If the amount of explanation is large, the recording can be divided into multiple sessions, and each session can be assigned to independent unit support information, thereby keeping each voice session below a predetermined time.

[0064] The unit support information generation device 130 can acquire multiple still images for a single audio recording. Specifically, after acquiring the first still image triggered by the start of audio input, subsequent still images may be acquired sequentially during or after audio input, triggered by additional shooting operations by the provider. The association unit 133 associates the acquired multiple still images with a single audio recording to the same unit identifier and generates it as a single unit support information. This makes it possible to record the subject being explained for a single audio recording as multiple still images for different angles, different magnifications, or work progress states.

[0065] The association unit 133 associates the acquired still images, short audio clips, identifiers, location information, time information, language information, and, if necessary, provider information or reference images, and generates them as unit support information.

[0066] The workflow candidate generation unit 136 may generate candidate presentation orders for the generated unit support information based on the recording order, positional relationship, time order, voice input order, equipment order, or process order.

[0067] The administrator may review the generated candidate workflows via the management interface unit 127 and edit the branching conditions, priority, skip conditions, re-presentation conditions, etc., to finalize them as workflow support information.

[0068] This invention can be applied, for example, to equipment maintenance, where still images and short audio clips are presented sequentially for each inspection point, and the system transitions to unit support information corresponding to the next inspection point as the worker moves.

[0069] The present invention can also be applied to activation methods that do not depend on location information. For example, after an operator activates a desired task from a QR code, calendar, URL, work sheet, or list of tasks, short audio clips and still images set for each task unit are presented sequentially, and the operator can transition to the next process upon completion of the audio instruction and audio playback. This allows the operator to perform tasks in a manner close to so-called eyes-free and hands-free, minimizing screen and manual operations while keeping their eyes and hands focused on the task at hand. In particular, when performing tasks while moving or using their hands, the operator can proceed with the task primarily using auditory guidance via short audio clips, and refer to still images when necessary for confirmation. Since the display of still images is maintained even after the audio ends, the operator can listen to the audio while keeping their hands and eyes focused on the task at hand, and return their gaze to the still image as needed to compare it with the actual object.

[0070] This section describes an application example to valve inspection work on factory equipment. The system voice asks, "Can you explain the question while holding your finger over the object in front of you?" In response, a veteran worker records their voice within 15 seconds while holding their finger over the valve and saying, "Please confirm that the handle at the red mark is fully open." Upon starting the voice input, a still image of the valve is automatically captured, and a unit support information is generated. Similarly, unit support information is generated for each check point, such as "pressure gauge check" and "leak check." Later, when a new worker reads a QR code near the same valve, the identifier receiver 111 acquires equipment identification information, and the presentation control unit 114 starts a workflow based on the veteran's voice and still image. The worker can proceed through the presentations at their own pace using voice or touch operation.

[0071] In the valve inspection example described above, the provider can associate a single voice command, "Please confirm that the handle at the red mark is fully open," with a total of three still images: a still image showing the entire valve, a still image zoomed in on the handle, and a still image zoomed in on the pressure gauge. The worker can listen to the voice command and switch between the three still images as they see fit using a swipe or voice command. The voice command is not interrupted by the switching of still images, allowing the worker to sequentially refer to multiple points of inspection while listening to the explanation. Furthermore, even when the worker is concentrating their attention on a task or working while moving, the camera unit 118 can compare the acquired image with the reference image at a predetermined timing, and automatically transition to the next unit support information when the degree of matching reaches a predetermined threshold. This allows the worker to receive information according to the progress of their work without requiring any screen operation.

[0072] Furthermore, the present invention can also be applied to equipment preparation, item verification, or patrol verification in medical settings; work verification for each field, verification of spraying or harvesting procedures in agriculture; on-site verification and equipment verification in disaster prevention activities; and construction procedure verification, hazardous area verification, or inspection point verification at construction sites.

[0073] The information presentation system 100 of this embodiment can also be applied to patrol inspections of construction sites. The site supervisor walks along the scaffolding of the construction site and records voice at each checkpoint. For example, at checkpoint P1, the provider records a voice saying, "Please check if this brace is loose," and the image acquisition unit 131 automatically captures a still image of the checkpoint when voice input begins. Similarly, unit support information is generated at each of the checkpoints P2 to P5. The workflow support information defines the patrol order of checkpoints P1 to P5 and each transition condition. Later, when another worker walks this patrol route, the voice of the senior supervisor and a still image are presented each time they approach a checkpoint.

[0074] The information presentation system 100 of this embodiment can also be applied to shift handovers in nursing care facilities. Daytime caregivers record a voice message of 15 seconds or less in front of each resident's room, giving instructions regarding the resident. For example, in front of room R1, they might record, "Please speak to Resident XX from the right side. He has difficulty hearing with his left ear," and a still image showing the entrance to the room is automatically captured when the voice input begins. When a nighttime caregiver approaches the same room R1, an identifier is obtained by reading location information or an NFC tag attached to the room, and the voice message and still image from the daytime caregiver are displayed.

[0075] Because the support information for each unit is independent, it is easy to replace only a portion of the process, replace equipment by target, replace by language, or replace on a site-by-site basis.

[0076] Furthermore, workflow support information can be managed not only as a single, complete entity, but also by combining multiple workflow support information components. For example, basic workflow support information corresponding to common processes and workflow support information corresponding to additional processes depending on the situation can be combined and used as a series of presentation sequences. It is also possible to replace and reconfigure basic workflow support information with other workflow support information, and workflow support information corresponding to common finishing processes can be added as a common component to multiple combinations. The management interface unit 127 allows for the combination, replacement, and reconfiguration of multiple workflow support information on the management screen, and the workflow candidate generation unit 136 can generate candidate presentation sequences based on the combined workflow support information. This enables flexible work support in response to changes in the field and situation.

[0077] In a configuration that utilizes the API provision unit 128, the management device 120 may cooperate with an external maintenance system, process management system, inventory management system, sensor network, or work record system to provide unit support information corresponding to the target equipment status or process status to the presentation terminal 110.

[0078] When using multiple display terminals 110, each display terminal 110 may independently receive an identifier and acquire different unit support information and workflow support information. This allows multiple workers to perform different processes in parallel within the same work site.

[0079] When a worker approaches a target piece of equipment, the system begins displaying still images and short audio clips of unit support information associated with that equipment. At this time, multiple candidate units are extracted based on the current location, and one unit support information is selected based on spatial range information and priority. If multiple unit support information is ordered, the display of subsequent unit support information is permitted only on the condition that the worker passes through the starting unit support information, thus maintaining consistency in the display order even if the worker enters from an intermediate point. When the worker leaves the departure detection range, the unit support information is updated to a displayed state, and re-display is set as necessary. In addition, unit support information located outside the departure detection range of the currently displayed unit support information is excluded from display and skipped, so that only information along the worker's movement path is displayed. When the worker moves to another target location, the system transitions to the next candidate unit, skips previously displayed units, or re-displays information upon reappearance. Furthermore, the entire movement-synchronous generation and display process involves sequentially executing the acquisition of still images and short audio clips, association with location information, candidate extraction, priority control, display, departure detection, and re-display control.

[0080] Figure 5 shows an example of the data structure of unit support information. As shown in Figure 5, unit support information may consist of a unit identifier, a still image, audio (short audio), provider information, a reference image, location information, time information, and identification information. Of these, the still image and audio are essential components, while the provider information, reference image, location information, time information, and identification information are optional components that may be included as needed.

[0081] Figure 6 shows an example of a data structure for workflow support information. As shown in Figure 6, workflow support information may be configured as a data structure that includes a unit identifier, process number or confirmation point number, presentation order, transition method, and branch destination. For example, U001 is a manual transition in presentation order 1, U002 is an automatic transition in presentation order 2, and U003 is a conditional transition in presentation order 3 with a branch set to U004 or U005, where U004 and U005 are configured as manual and automatic transitions as the processes after the branch, respectively. In this way, workflow support information allows for the flexible definition of not only linear sequence presentation but also conditional branch presentation.

[0082] Figure 7 shows an example of a presentation screen including a still image, provider information, and a reference image. As shown in Figure 7, the presentation screen may consist of a still image display area, a reference image display area, a provider information display area, an audio playback status display, and a next button. The process number or confirmation point number is displayed at the top of the screen. A still image indicating the work target or confirmation point is displayed in the still image display area, and the reference image is placed near the still image and can be referenced as needed. Provider information is displayed at the bottom of the screen, allowing the worker to proceed with the work while confirming the source of the information. Since the display of the still image is maintained during and after audio output, it becomes easier for the worker to use the audio content as a clue to confirm the physical object and continue the work. Transition operations are primarily voice-operated, but manual transitions can also be performed by pressing the next button via touch operation. Furthermore, playback can be completed by the worker's operation even while audio playback is in progress.

[0083] Figure 8 shows an example of the basic flow of the information presentation process. In the identifier reading step, the identifier receiving unit 111 receives and reads the identifier. Next, in the unit support information acquisition step, the unit support information acquisition unit 112 acquires multiple unit support information corresponding to the identifier. Next, in the workflow support information acquisition step, the workflow support information acquisition unit 113 acquires the corresponding workflow support information. Next, in the still image display step, the presentation control unit 114 displays the still image included in the selected unit support information on the display unit 116. Next, in the audio output step, the audio output unit 117 outputs audio. The display of the still image is maintained during and after audio output. Next, in the transition condition determination step, the transition control unit 115 determines the transition method defined in the workflow support information. If the transition method is a manual transition, the transition control unit 115 executes the transition triggered by a manual operation by the operator. If the transition method is an automatic transition, the transition control unit 115 executes the transition triggered by the completion of audio playback. If the transition method is a conditional transition, the transition control unit 115 executes the transition when a predetermined condition is met. In any transition method, if the next unit support information exists (YES), the process returns to the still image display step; if it does not exist (NO), the process is completed and terminated.

[0084] Figure 9 shows an example of the unit support information generation process flow. The provider stands in the workspace with the work object in front of them and starts voice input. The voice acquisition unit 132 acquires the voice information entered by the provider. When the start of voice input is detected, the image acquisition unit 131 automatically acquires a still image of the work object. The association unit 133 associates the acquired voice and still image as a pair. The unit generation unit 135 generates unit support information including the voice and still image and stores it in the storage unit 122. The provider can sequentially generate multiple units of support information by repeating the same operation for the next work object or confirmation point.

[0085] This generation process can also be applied to worker recording. Specifically, when the presentation control unit 114 presents a unit support information, and the worker uses the presented content as a clue to confirm the actual object while making voice input, the voice acquisition unit 132 acquires the voice, and the image acquisition unit 131 automatically captures a still image triggered by the voice input. The association unit 133 and the unit generation unit 135 generate new unit support information associated with the presented unit support information, using the worker's voice and the still image. This allows for the layered accumulation of the worker's understanding, confirmation results, or insights on top of the knowledge of the preceding provider.

[0086] Figure 10 shows the types of transition methods. Manual transitions are triggered by manual operation by the operator, and the system transitions to the next unit support information when the operator explicitly instructs the transition by touch operation or the like. Automatic transitions occur automatically without manual operation by the operator, and one example is a transition to the next unit support information triggered by the completion of audio playback by the audio output unit 117. Furthermore, another example of an automatic transition is a conditional transition, which occurs when a predetermined condition is met. In a conditional transition, for example, an image comparison process is performed between the actual image acquired by the imaging unit 118 and a still image or reference image included in the currently displayed unit support information, and the system transitions to the next unit support information when the degree of agreement between the two exceeds a predetermined threshold.

[0087] Figure 11 shows the processing flow during offline operation. In the communication feasibility determination step, the presentation terminal 110 determines whether communication with the management device 120 is possible. If communication is possible (YES), the presentation terminal 110 obtains unit support information and workflow support information from the management device 120 and stores them as a local cache in the local storage unit 119. If communication is not possible (NO), the presentation terminal 110 determines whether a cache exists in the local storage unit 119. If a cache exists (YES), the presentation terminal 110 reads the cache from the local storage unit 119. If a cache does not exist (NO), the presentation terminal 110 cannot continue presentation. In either path, in the presentation control continuation step, the presentation control unit 114 continues presentation, and in the transition control continuation step, the transition control unit 115 continues transition control. After the transition control continuation, the process returns to the communication feasibility determination step, and the presentation terminal 110 repeats the process.

[0088] In the steps of continuing presentation control and continuing transition control, the presentation terminal 110 continues presentation and transition control even when communication is interrupted or in environments where external positioning signals are unstable, by referring to the information stored in the local memory unit 119. At this time, the operator can identify the target based on their own judgment, regardless of the presence or absence of external infrastructure, by comparing the still image contained in the cached unit support information with the real scene in front of them, and complete the task.

[0089] Figure 12 shows an example of the configuration of the management screen. The management interface unit 127 provides a management screen that includes a unit support information list display area, a workflow editing area, and a unit details display area. In the unit list display area, each unit support information is displayed in a list along with the unit identifier, still image, and audio, and the administrator can add, delete, and replace the unit support information. In the workflow editing area, the transitions from U001 to U002, U002 to U003, U003 to U004, or U005 are displayed as a flow, and the administrator can change the presentation order, set branches, and manage the history. In the unit details display area, the ID, title, description, attached still image file, and audio file of the selected unit support information are displayed, and the administrator can edit them.

[0090] Figure 13 shows an example of how provider information is displayed. Provider information may include the provider's photograph, name, affiliation, qualifications, years of experience, and creation date, and may also display a certification status indicating that the information is certified. A provider is someone who has experience or knowledge regarding the work object or work process in question. By referring to the provider information on the display screen, the worker can confirm what kind of experience and qualifications the person who created the unit support information has, and whose voice the audio is. This allows the worker to proceed with the work while receiving information based on the experience and judgment of a specific person, while also incorporating their own judgment. Since the worker performs the work by checking the actual product while referring to the provider's ideas, the information display does not replace the worker's judgment. Rather, the asynchronous presentation of the knowledge of an experienced provider functions as support for the worker to perform the work while making their own judgments.

[0091] Figure 14 shows the history management and analysis screen. The upper left of the screen has filters for date and time range, worker, and unit ID, and the filtering conditions can be removed by deleting the filters. The presentation history list displays the date and time, worker, unit ID, required time, and status, allowing the administrator to check the presentation status of each unit support information in chronological order. The analysis graph area displays the completion rate, average required time, and number of re-presentations for each unit support information as graphs. The summary area displays the total number of units, average efficiency, and total work time as aggregated values. This allows the administrator to understand the progress of the entire workflow and the proficiency level and areas of difficulty of each unit support information, and to use this information to improve unit support information and workflow support information.

[0092] Figure 15 shows the presentation of identifier candidates through location information linkage. The location linkage unit 126 extracts identifier candidates associated with the location based on the location information acquired from the presentation terminal 110. The presentation terminal 110 displays the worker's current location on the floor map and also displays nearby equipment as identifier candidates such as EQ-001, EQ-002, EQ-003, along with distance information. By selecting the target equipment from the presented identifier candidates, the worker can start acquiring corresponding unit support information and workflow support information based solely on location information, without requiring barcode or QR code reading operations.

[0093] Figure 16 shows the flow of unit support information generation in a mobile synchronization type. The provider moves around the workspace while carrying a mobile terminal equipped with GPS. When the provider starts voice input at point P1, the voice acquisition unit 132 acquires voice information, and the image acquisition unit 131 automatically acquires a still image of that point upon detection of the start of voice input. The position and time acquisition unit 134 acquires position information L1 and time information T1 at the time of acquisition. The unit generation unit 135 generates unit support information U1 as a set of voice, still image, position information and time information. As the provider moves sequentially to points P2 and P3 and repeats the same operation, unit support information U2 and U3 are generated sequentially. Once recording is complete at all points, the workflow candidate generation unit 136 generates workflow support information candidates based on the provider's movement order and position information. Note that this generation method is an example of mobile recording, but it can also be applied to non-mobile recording.

[0094] In this specification, "still image" means a fixed image that does not change continuously over time, and does not include moving images or looping images whose content changes continuously, at least during the presentation of unit support information. In other words, a still image may be a single image, a selected representative image, or a fixedly displayed reference image, and may remain displayed even after the audio ends until the next transition or operator action is taken. Even if a single piece of unit support information includes multiple still images, each image is a fixed image (still image) that is independently and individually captured, and does not constitute a moving image.

[0095] The information presentation system 100 combines short audio recordings of the provider's voice at the worksite with still images, allowing workers to proceed with their work at their own pace while confirming the actual object without face-to-face instruction. Furthermore, since the display of the still image is maintained even after the audio ends, the load on the worker's working memory is reduced, enabling both object confirmation and information retention. Moreover, in the first mode, transitions are initiated by the worker's active operation, while in the second mode, transitions are triggered automatically when transition conditions such as location information or image comparison results are met. Since both modes are switchable, flexible operation is possible according to the nature of the work and the worksite environment.

[0096] Unit support information and workflow support information are stored as separate, mutually referential data, and changes to workflow support information can be updated independently without requiring the regeneration of still images and audio included in unit support information. This allows for efficient replacement, language-specific support, and process changes on a site-by-site basis, and ensures that the knowledge of experienced providers is continuously provided to workers even when those providers are absent.

[0097] Each of the above functions may be implemented by a dedicated circuit, or by a computer executing a program.

[0098] In this case, the program enables a computer to implement the functions of the presentation terminal 110, the management device 120, or the unit support information generation device 130, and can be provided by recording it on a non-temporary computer-readable recording medium.

[0099] The present invention is not limited to the embodiments described above and can be modified as appropriate without departing from its spirit. The components of each embodiment may be used individually or in combination as appropriate.

[0100] The configurations, processing sequences, names, and numerical ranges described herein, in the claims, abstract, and drawings are interchangeable to the extent that they do not contradict each other.

[0101] This invention can be widely applied to industrial fields such as equipment maintenance, inspection, assembly, construction, and transportation. In addition, it can be applied to equipment preparation, item verification, or patrol verification in medical settings, field-by-field work procedure verification in agriculture, on-site verification, equipment verification, or initial response support in disaster prevention, and construction procedure verification, hazardous area verification, or inspection point verification in construction.

[0102] 100 Information presentation system 110 Presentation terminal 111 Identifier receiving unit 112 Unit support information acquisition unit 113 Workflow support information acquisition unit 114 Presentation control unit 115 Transition control unit 116 Display unit 117 Audio output unit 118 Shooting unit 119 Local storage unit 120 Management device 121 Communication unit 122 Storage unit 123 Control unit 124 Translation processing unit 125 History management unit 126 Location linkage unit 127 Management interface unit 128 API provision unit 130 Unit support information generation device 131 Image acquisition unit 132 Audio acquisition unit 133 Association unit 134 Location / time acquisition unit 135 Unit generation unit 136 Workflow candidate generation unit 137 Output control unit

Claims

1. An information presentation system comprising: an identifier receiving means for receiving an identifier; a unit support information acquiring means for acquiring a plurality of unit support information, including at least a still image and audio of a predetermined length or less, based on the identifier; a workflow support information acquiring means for acquiring workflow support information that defines the presentation order, branching conditions, transition conditions, and priority of the plurality of unit support information; a presentation control means for displaying the still image included in the selected unit support information and outputting the audio; and a transition control means for controlling transitions between the plurality of unit support information based on the workflow support information, wherein the unit support information and the workflow support information are stored as separate data that can refer to each other, and changes to the workflow support information can be updated independently in a manner that does not require the regeneration of the still image and audio included in the unit support information; and the presentation control means, in a first mode, after the output of the audio ends, accepts the next operation input from the operator while maintaining the display of the still image and presents the next unit support information in accordance with the next operation input; and in a second mode, after the output of the audio ends, if a transition condition including location information or the result of comparing captured images is met, presents the next unit support information without requiring the next operation input.

2. The information presentation system according to claim 1, wherein the unit support information is configured to correspond to a single work unit or confirmation point and includes at least one still image and at least one audio.

3. The information presentation system according to claim 1, wherein the unit support information includes a plurality of still images corresponding to a single sound, and the presentation control means has a plurality of still image switching function that selects and displays one of the plurality of still images and switches the display target to another still image according to the operator's operation.

4. The information presentation system according to claim 3, wherein the switching operation in the multiple still image switching function includes at least one of a swipe operation, a voice command, and a button operation, and the presentation control means does not interrupt the playback of the voice even when the switching operation is performed.

5. The information presentation system according to claim 1, wherein the unit support information is associated with provider information that can identify the provider who created the unit support information, and the presentation control means presents the provider information when the unit support information is presented.

6. The information presentation system according to claim 1, wherein the audio included in the unit support information is recorded by a provider who has experience or knowledge of the work object or work process, while standing in front of the work object or while moving around the work space, and the presentation control means outputs the audio to the worker at a time different from the time the provider recorded it.

7. The information presentation system according to claim 5, wherein the provider is a person who has a predetermined relationship with the worker, and the predetermined relationship includes at least one of the following: a relationship belonging to the same work site or work area, a relationship belonging to the same job or occupation, a relationship designated by the worker, or a relationship recommended by the manager.

8. The information presentation system according to claim 1, wherein the transition control means automatically transitions from the currently presented unit support information to the next unit support information when predetermined conditions are met, based on the comparison result between the still image or reference image and the image acquired by the shooting unit.

9. The information presentation system according to claim 1, wherein the transition control means automatically transitions from the currently presented unit support information to the next unit support information when a predetermined condition is met, based on the result of matching sensing information, which includes at least one of position, movement, speech content, gaze, biometric information, and the state of an object, obtained from a sensor provided on the presentation terminal or an external sensor, with predetermined condition information associated with the unit support information.

10. The information presentation system according to claim 1, wherein the presentation control means is configured to be switchable between a first mode and a second mode, and in the first mode, even if the transition conditions are met, it waits for operator input, and in the second mode, when the transition conditions based on sensing information are met, it presents the next unit support information without operator input.

11. The information presentation system according to claim 1, wherein the transition control means controls the automatic transition between the unit support information based on state information including equipment status, process status, or sensor values ​​based on external data.

12. The information presentation system according to claim 1, wherein when the worker who has received the unit support information from the presentation control means inputs voice while confirming the actual object using the presented content as a clue, the system automatically captures a still image of that location as a trigger for the voice input and generates new unit support information including the worker's voice and the still image.

13. The information presentation system according to claim 1, wherein the transition control means selects a target for presentation based on spatial range information and priority when multiple unit support information items are candidates for presentation.

14. The information presentation system according to claim 13, wherein the spatial range information includes at least one of a presentation start range and a presentation continuation range, and a departure determination range, and the transition control means updates the unit support information to a presented state, excludes it from presentation candidates, or sets re-presentation conditions in response to departure from the departure determination range.

15. The information presentation system according to claim 14, wherein the transition control means, when the position information corresponds to the spatial range information again after departure, re-presents the unit support information or transitions to another candidate unit support information.

16. The information presentation system according to claim 8, wherein the comparison result includes at least one of pixel difference, feature matching, and matching degree determination by a trained model.

17. The information presentation system according to claim 1, wherein the transition control means enables the execution of at least one of the following: an input operation that accepts at least one of the following, which the operator has determined on their own based on the comparison of the still image with the actual object: proceeding to the next unit, returning to the previous unit, re-presenting, or re-playing, the input operation including at least one of contactless voice input, gesture input, gaze input, and sensing information acquired from a presentation terminal or external sensor; and a transition operation that recognizes the content of the operator's voice response to the voice inquiry and is performed based on the result of the recognition.

18. The information presentation system according to claim 1, comprising: a presentation terminal that displays the still images and outputs the sound; and a management device having a storage unit for storing the unit support information and the workflow support information, wherein the presentation terminal has a local storage unit for storing at least a portion of the unit support information and the workflow support information, and retrieves information from the local storage unit to continue presentation when communication is restricted.

19. The information presentation system according to claim 1, wherein the unit support information includes text or audio corresponding to multiple languages, and the presentation control means switches the display or output content according to the language information.

20. The information presentation system according to claim 1, wherein the identifier includes at least a portion of the target equipment identification information, work target identification information, location identification information, process identification information, user identification information, and language information.

21. The information presentation system according to claim 1, wherein multiple presentation terminals each independently receive identifiers and present information based on different unit support information and workflow support information.

22. A unit support information generation device comprising: voice acquisition means for receiving voice input from a provider who has experience or knowledge of the work object or work process; image acquisition means for detecting the start of the voice input and, triggered by the detection of the start, acquiring a still image indicating the work object or location; association means for associating the acquired still image and the voice data based on the voice input with the same unit identifier; and generation means for generating unit support information based on the association by the association means.

23. The unit support information generation device according to claim 22, wherein the association means associates, in addition to the still image and the sound, an identifier, location information, time information, language information, provider information, and at least a portion of the reference image.

24. The unit support information generation device according to claim 22, wherein the image acquisition means acquires a second and subsequent still image in addition to the still image acquired when the input of the voice begins, triggered by an additional shooting operation during or after the input of the voice, and the association means associates the acquired multiple still images with the same unit identifier as a single voice.

25. The unit support information generation device according to claim 22, further comprising a workflow candidate generation means for generating candidate workflow support information based on a plurality of unit support information, wherein the generation of candidate workflow support information by the workflow candidate generation means can be performed independently of the generation of unit support information, and the setting or modification of transition conditions, presentation order, or branching conditions included in the candidate workflow support information is performed in a manner that does not require the regeneration of still images and audio included in the unit support information.

26. The unit support information generation device according to claim 25, wherein the workflow candidate generation means generates a presentation order or branching candidate based on at least one of location information, time information, voice input order, equipment order, or process order associated with the plurality of unit support information.

27. A program for causing a computer to function as one of the means described in any one of claims 1 to 21.

28. A non-temporary computer-readable recording medium on which the program described in claim 27 is recorded.

29. A program for causing a computer to function as one of the means described in any one of claims 22 to 26.

30. A non-temporary computer-readable recording medium on which the program described in claim 29 is recorded.

31. An information presentation method comprising: receiving an identifier; acquiring a plurality of unit support information, including at least a still image and audio of a predetermined length or less, based on the identifier; acquiring workflow support information that defines the presentation order, branching conditions, transition conditions, and priority of the plurality of unit support information; displaying the still image included in the selected unit support information and outputting the audio; and controlling the transitions between the plurality of unit support information based on the workflow support information, wherein the unit support information and the workflow support information are stored as separate data that can refer to each other, and changes to the workflow support information can be updated independently in a manner that does not require the regeneration of the still image and audio included in the unit support information; in the first mode, after the output of the audio ends, the display of the still image is maintained and a next operation input from the operator is accepted, and a transition in the step of controlling the transition in accordance with the next operation input is executed; and in the second mode, after the output of the audio ends, if a transition condition including location information or a comparison result of a captured image is met, a transition in the step of controlling the transition is executed without requiring the next operation input.

32. The information presentation method according to claim 31, further comprising the step of maintaining the display of the still image at least until the next transition or operator operation is performed, even after the output of the sound has been completed.

33. The information presentation method according to claim 31, wherein the step of displaying a still image includes a switching step of switching the still image to be displayed in accordance with the operator's operation when a single unit support information includes multiple still images, and the switching step is performed without interrupting the output of the sound.

34. The information presentation method according to claim 31, wherein the step of controlling the transition includes a step of executing an automatic transition when a predetermined condition is met, based on at least one of the following: the result of comparing the still image or reference image with the captured image, the result of matching the position information with the spatial range information, or state information based on external data.

35. The information presentation method according to claim 34, wherein the step of controlling the transition includes a step of selecting a target to be presented based on spatial range information and priority when multiple unit support information items are candidates for presentation; a step of updating the presented status or setting re-presentation conditions in response to departure from the departure determination range; and at least one of the following steps by the operator: proceeding to the next unit, returning to the previous unit, re-presenting, or replaying.

36. A method for generating unit support information, comprising: acquiring audio information as audio input by a provider having experience or knowledge of the work object or work process while facing the work object or moving through the work space; detecting the start of the input of the audio information and automatically capturing a still image showing the state of the work object or location as a trigger for such detection; associating the still image with the audio; and generating unit support information based on the association.

37. The unit support information generation method according to claim 36, wherein the association step includes associating at least a part of an identifier, location information, time information, language information, provider information, and a reference image, in addition to the still image and the audio.

38. A method for generating unit support information according to claim 36, further comprising the step of generating candidate workflow support information based on a plurality of generated unit support information, wherein the step of generating candidate workflow support information can be performed independently of the step of generating unit support information, and the setting or modification of transition conditions, presentation order, or branching conditions included in the candidate workflow support information is performed in a manner that does not require the regeneration of the content of still images and audio included in the unit support information.

39. The method for generating unit support information according to claim 38, wherein the step of generating candidate workflow support information is to generate a presentation order or branching candidate based on at least one of location information, time information, voice input order, equipment order, or process order associated with the plurality of unit support information.

40. A program for causing a computer to perform each of the steps described in any one of claims 31 to 39.

41. A non-temporary computer-readable recording medium having the program described in claim 40 recorded on it.

Images