Video correction method and video correction device

Abstract

In this video correction method and this video correction device, when processing an image in which an action of an actor has been imaged at an angle of view that is wider than a prescribed angle of view possessed by a person's field of view, a place of attention that appears in the video and pertains to the action is specified, and a prescribed region that includes the place of attention therein is extracted. The image is then corrected by correction processing in which the prescribed region is treated as a simply-connected region corresponding to an angle of view that is narrower than the prescribed angle of view, and the corrected image is outputted.

Description

Image correction method and image correction device

[0001] The present disclosure relates to an image correction method and an image correction device.

[0002] In order to effectively support work training, a technology has been disclosed that effectively presents the trainee with the results of an evaluation of the work content of the worker (trainee), as well as the locations and causes of work defects (see Patent Document 1). This technology collects information on the worker's gaze video and gaze position, detects the work object, and generates a time-series gaze transition map that represents the transition pattern of the worker's gaze position over time based on the positional relationship between the work object and the gaze position. The technology then compares the time-series gaze transition map generated for the trainee with the time-series gaze transition map generated for a worker performing a model work, and outputs the results.

[0003] Japanese Patent Application Laid-Open No. 2021-043561

[0004] According to the technology described in Patent Document 1, when a trainee uses an image captured with a wider angle of view than the human field of view to learn an action, there is a problem in that the trainee's understanding of the action is hindered due to image distortion.

[0005] The present disclosure has been made in view of the above-mentioned problems, and aims to provide an image correction method and an image correction device that can present a training image to a trainee that does not hinder the trainee's understanding of a movement, even when the trainee uses images captured with a field of view wider than the human field of view to learn a movement.

[0006] In order to solve the above-mentioned problems, an image correction method and an image correction device according to one aspect of the present disclosure process an image of a performer's action captured at a field of view wider than a predetermined field of view of a human visual field, identify a point of interest related to the action that appears in the image, extract a predetermined area that includes the point of interest, correct the image by performing a correction process that converts the predetermined area into a simply connected area corresponding to a field of view narrower than the predetermined field of view, and output the corrected image.

[0007] According to the present disclosure, even when a trainee uses images captured with a wider angle of view than the human field of view to learn a movement, the trainee can be presented with training images that do not hinder the trainee's understanding of the movement.

[0008] Fig. 1 is a block diagram showing a configuration of an image correction device according to an embodiment of the present disclosure; Fig. 2 is a flowchart showing processing by an image correction device according to an embodiment of the present disclosure; Fig. 3 is a diagram showing an example of an image before correction; Fig. 4 is a diagram showing an example of an image after correction; Fig. 5 is a diagram showing an example of an image and a figure superimposed on the image.

[0009] Next, embodiments of the present disclosure will be described in detail with reference to the drawings. In the description, the same components are designated by the same reference numerals and redundant description will be omitted.

[0010] [Configuration of Image Correction Device] Fig. 1 is a block diagram showing the configuration of an image correction device according to an embodiment of the present disclosure. As shown in Fig. 1, the image correction device includes an acquisition unit 71, a controller 100, and an output unit 400. In addition, the image correction device may include a database 73.

[0011] The acquisition unit 71 acquires an image capturing the movement of the performer at a wider angle of view than the predetermined angle of view of a human field of view. For example, the acquisition unit 71 may include an imaging device capable of capturing an image of the surroundings at a wider angle of view than the predetermined angle of view of a human field of view, such as an omnidirectional multi-camera, a spherical camera, a fisheye camera, or a wide-angle camera. The imaging device may be attached to the performer or may be installed around the performer.

[0012] In this embodiment, the "actor" may be, for example, a worker performing a task in a manufacturing process, or an athlete performing an action in various sports. The actions of the "actor" are acquired by the acquisition unit 71, and the video of the recorded actions is presented to the trainee, allowing the trainee to learn the actions of the actor. As a result, the trainee can understand the tasks in the manufacturing process and other movements during a sport. The "actor" is not limited to the examples given here.

[0013] The image acquired by the acquisition unit 71 may be an image generated by stitching together images acquired by a plurality of imaging devices, or may be a panoramic image.

[0014] The acquisition unit 71 may also acquire gaze information indicating the gaze of the performer. For example, the acquisition unit 71 may include a sensor that performs gaze sensing to capture an image of the performer's eyeballs and acquire the gaze direction.

[0015] Furthermore, the acquisition unit 71 may acquire posture information indicating the posture of the performer. For example, the acquisition unit 71 may include an inertial sensor worn by the performer. The acquisition unit 71 may also be a system that measures the posture and movement of the performer by applying acceleration, angular velocity, and direction information obtained from the inertial sensor worn by the performer to a skeletal model. Alternatively, the acquisition unit 71 may calculate posture information by analyzing movement information obtained by wearing body markers or the like on the athlete's body.

[0016] The acquisition unit 71 may also include a force sensor attached to the body of the performer. The force sensor resolves the physical force applied to the force sensor or the torque, which is the moment of the force, into components in the X, Y, and Z directions and converts them into electrical signals for detection. In this way, the acquisition unit 71 may acquire the magnitude of the force or moment associated with the performer's movement.

[0017] For example, a force sensor is a device that acquires the degree of force exerted by an actor. Pressure-type wearable gloves or electromyographic bands may be attached to the actor's fingers or limbs, and the degree of force exerted can be acquired, which cannot be determined by the actor's appearance.

[0018] The database 73 stores a document describing procedures related to the actions of the performer. The database 73 may store a work procedure manual as a document describing procedures related to the actions of the performer. The document may include nouns indicating the names of objects that are the targets of the actions, the names of tools used in the actions, etc. The document may also include verbs indicating the content of the actions performed on the objects and tools.

[0019] For example, a document may include a description regarding the direction of the actor's gaze. More specifically, if a document includes a description "check the order," the description may be treated as a description regarding the direction of the actor's gaze, based on the inclusion of the verb "check."

[0020] Furthermore, the noun "order" that appears in conjunction with the verb mentioned above can be treated as an object located in the direction of the gaze. In this way, based on the description "check XXX", it can be seen that the actor is directing their gaze toward the object identified by "XXX". The description of the actor's gaze direction contained in the document is not limited to the examples given here.

[0021] The document may also include a description of the movement of a specific part of the performer's body. More specifically, if the document includes descriptions such as "pinch XXX," "apply XXX," "assemble XXX," "temporarily tighten XXX," and "set XXX," these descriptions may be treated as descriptions of the movement of a specific part of the performer's body on the basis that they contain the verbs "pinch," "apply," "assemble," "temporarily tighten," and "set."

[0022] Furthermore, the object identified by the noun "xxx" that appears in association with the above verbs may be treated as the object of the action. The description of the movement of a specific part of the actor's body contained in the document is not limited to the examples given here.

[0023] Additionally, the database 73 may store various types of information acquired via the acquisition unit 71. For example, the database 73 may store images of the movements of the performer, line of sight information, posture information, the magnitude of the force related to the movements of the performer, the magnitude of the moment, and the like.

[0024] The output unit 400 outputs an image corrected by the controller 100, which will be described later. For example, the output unit 400 is a display device such as a computer screen or a television LCD screen. The output unit 400 may also be a display device used in a mobile phone, a portable game console, a calculator, a clock, or the like. Examples of the display device include a display (monitor) and a projector. Examples of the display include a liquid crystal display (LCD), an organic light-emitting diode display (OLED), a plasma display (PDP), a micro LED display (mLED), a video projector, a cathode ray tube (CRT), and the like. The output unit 400 is not limited to the examples given here.

[0025] The controller 100 processes the image acquired by the acquisition unit 71. For example, the controller 100 is a general-purpose computer equipped with a CPU (Central Processing Unit), memory, and input / output units. A computer program (image correction program) for causing the computer to function as an image correction device is installed in the computer. By executing the computer program, the computer functions as multiple information processing circuits provided in the image correction device.

[0026] The image correction program may be stored in a storage medium such as a non-transitory computer-readable medium, or may be distributed via a telecommunications line.

[0027] While the following describes an example in which the multiple information processing circuits provided in the image correction device are realized by software, it is of course also possible to configure the information processing circuits by providing dedicated hardware for executing each of the information processes described below.Furthermore, the multiple information processing circuits may be configured as individual hardware.

[0028] The controller 100 includes, as examples of multiple information processing circuits (information processing functions), a focus point identification unit 110, a specified area extraction unit 120, an image correction unit 130, a document analysis unit 140, an eye line information acquisition unit 150, and a posture information acquisition unit 160.

[0029] The attention location identifying unit 110 identifies an attention location related to the action of the performer that appears in the image. For example, the attention location identifying unit 110 may identify the target of the action of the performer as the attention location. Alternatively, the attention location identifying unit 110 may identify an attention location by referring to a document that describes a procedure related to the action. The attention location identifying unit 110 may identify an attention location based on a noun or verb included in the document.

[0030] More specifically, the interest portion identification unit 110 may identify an area on an image corresponding to a noun included in a document as the interest portion. Here, the interest portion identification unit 110 may extract an area on an image corresponding to a noun by object recognition using machine learning, and identify the extracted area as the interest portion.

[0031] The interest portion identification unit 110 may identify an area on an image related to a verb included in a document as the interest portion. The interest portion identification unit 110 may also identify an object identified by a noun that appears in association with the verb as the interest portion.

[0032] Furthermore, when a document contains a description regarding the direction of the performer's line of sight, the attention location identifying unit 110 may extract the direction of the performer's line of sight based on the image.The attention location identifying unit 110 may also identify an object located in the direction of the performer's line of sight as the attention location.

[0033] Furthermore, when the document contains a description of the movement of a specific part of the performer's body, the attention part identifying unit 110 may extract the specific part based on the image. Then, the attention part identifying unit 110 may identify an area in which the specific part moves as the attention part.

[0034] When the movement of the performer includes a plurality of different steps, the attention spot identifying unit 110 may identify an attention spot for each step. Alternatively, when the target of the movement of the performer transitions, the attention spot identifying unit 110 may identify a transition source target and a transition destination target as attention spots.

[0035] The attention point identification unit 110 may correct the attention point based on line-of-sight information indicating the line of sight of the performer. For example, there are cases where the performer does not necessarily look at the target object of the action when performing the action. In such cases, the performer may perform the action while looking at an instrument that displays an indicator related to the action. In such cases, the attention point identification unit 110 may identify an object located in the performer's line of sight as the attention point, instead of a line of sight identified from a work procedure manual or the like.

[0036] Furthermore, the attention point identification unit 110 may correct the attention point based on posture information indicating the posture of the performer. For example, there are cases where the performer does not necessarily look at the target object of the action when performing the action. In such cases, the performer may grasp the situation of the action based on his or her own posture. In such cases, the attention point identification unit 110 may identify, as the attention point, a specific part of the performer's body identified by posture information indicating the posture of the performer, instead of a attention point identified from a work procedure manual or the like.

[0037] The predetermined region extraction unit 120 extracts a predetermined region that includes a location of interest. Here, the predetermined region is a part of the image acquired by the acquisition unit 71, and is a region corresponding to a narrower angle of view than the predetermined angle of view of a human field of view. The image acquired by the acquisition unit 71 is captured at a wider angle of view than the predetermined angle of view of a human field of view, and when converted to a planar view, distortion becomes large. Because the predetermined region is a region corresponding to a narrower angle of view than the predetermined angle of view of a human field of view, distortion when the predetermined region is converted to a planar view can be smaller than distortion when the original image is converted to a planar view.

[0038] Furthermore, when multiple points of interest are identified, the predetermined region extraction unit 120 may extract a predetermined region that includes all of the points of interest. The predetermined region extraction unit 120 may group the points of interest based on their proximity on the image, and extract a predetermined region that includes a group included in a region that corresponds to an angle of view narrower than the predetermined angle of view of a human field of view.

[0039] The predetermined region extraction unit 120 may extract multiple predetermined regions. In this case, the points of interest included in different predetermined regions may not fall within a field of view narrower than the predetermined field of view of a human being. In other words, the points of interest may be grouped together within the range of the predetermined field of view of a human being, and a predetermined region may be extracted for each group.

[0040] The image correction unit 130 corrects the image by a correction process that converts the predetermined region into a simply connected region corresponding to a field of view narrower than the predetermined field of view. For example, the image correction unit 130 may use the correction process to separate the predetermined region from the boundary of the image. Alternatively, the image correction unit 130 may use the correction process to position the predetermined region at the center of the corrected image.

[0041] The correction process by the image correction unit 130 will be described with reference to Figs. 3A and 3B. Fig. 3A is a diagram showing an example of an image before correction. Fig. 3B is a diagram showing an example of an image after correction. Fig. 3A shows an example of a planar display of an image captured by an omnidirectional multi-camera attached to the top of a helmet worn by an operator. The image shown in Fig. 3A shows the operator installing a part.

[0042] 3A, assume that points of interest R1 and R2 are identified by the point of interest identification unit 110. At this time, the predetermined region extraction unit 120 extracts a predetermined region DR that includes both points of interest R1 and R2.

[0043] In Figure 3A, the extracted predetermined region DR is divided into two when the image is viewed two-dimensionally, and is not a single connected region on the image. If an image in which the predetermined region DR, which should originally be a single region, has been divided into multiple regions is presented to a trainee, the trainee will need to frequently move his or her eyes to focus on the region of interest on the image. As a result, the trainee may be unable to understand the actions reflected in the image.

[0044] Therefore, the image correction unit 130 performs a correction process on the image shown in Fig. 3A to generate the image shown in Fig. 3B. In Fig. 3B, the predetermined region DR is maintained as a simply connected region on the image even when the image is viewed two-dimensionally. In particular, the predetermined region DR is located at a distance from the boundary of the image. It can also be seen that the predetermined region DR is located approximately at the center of the image. For example, the predetermined region DR is located so as to include the center position of the image. This makes it less likely that the trainee will be hindered from recognizing the point of interest reflected in the predetermined region.

[0045] Furthermore, if the image is a panoramic image, the image correction unit 130 may perform a correction process to place a predetermined area in front of the corrected panoramic image. When the image is a panoramic image, the image of an object placed in front of the panoramic image is displayed with less distortion and easier to understand than an image of an object placed in a position other than the front of the panoramic image. In this way, by placing the predetermined area in front of the panoramic image, the trainee is less likely to be hindered from recognizing a point of interest reflected in the predetermined area.

[0046] Alternatively, the image correction unit 130 may superimpose a graphic indicating a point of interest on the corrected image. In particular, the image correction unit 130 may superimpose a graphic indicating a point of interest on the corrected image when it is determined that the document includes a description regarding the direction of the performer's line of sight.

[0047] Furthermore, the image correction unit 130 may superimpose a graphic indicating an area in which a specific body part of the performer moves on the corrected image. In particular, when it is determined that the document includes a description regarding the movement of a specific body part of the performer, the image correction unit 130 may superimpose a graphic indicating an area in which the specific body part moves on the corrected image.

[0048] 4 is a diagram showing an example of an image and a graphic superimposed on the image. In FIG. 4, an arrow A1 is superimposed on the corrected image as a graphic indicating a point of interest R1. Furthermore, an arrow A2 is superimposed on the corrected image as a graphic indicating a point of interest R2. In addition, a sector BE is superimposed on the corrected image as a graphic indicating an area where a specific part of the performer's body moves.

[0049] The document analysis unit 140 analyzes a document describing procedures related to the actions of an actor. For example, the document analysis unit 140 determines whether the document includes a description regarding the eye direction of the actor. To extract information from the document, the document analysis unit 140 may analyze the document using a method capable of syntactic analysis, such as a morphological analyzer or a transformer. Alternatively, the document analysis unit 140 may analyze the document based on conditions set by a person through annotation in advance.

[0050] The document may include nouns indicating the name of the object that is the target of the operation, the name of the tool used in the operation, etc. The document may also include verbs indicating the content of the operation to be performed on the object and the tool.

[0051] For example, if a document contains the statement "check the order," the document analysis unit 140 may treat the statement as a statement regarding the direction of the actor's gaze, based on the fact that the statement contains the verb "check."

[0052] Furthermore, the document analysis unit 140 may treat the noun "order" that appears in association with the above verb as an object located in the direction of the performer's gaze. In this way, based on the description "check XXX", it can be seen that the performer is directing their gaze toward the object identified by "XXX". The method by which the document analysis unit 140 determines the description regarding the performer's gaze direction is not limited to the example given here.

[0053] Furthermore, if a document contains descriptions such as "pinch XXX," "apply XXX," "assemble XXX," "temporarily tighten XXX," or "set XXX," the document analysis unit 140 may treat these descriptions as descriptions of the movement of a specific part of the performer's body, based on the fact that they contain the verbs "pinch," "apply," "assemble," "temporarily tighten," and "set."

[0054] Furthermore, the document analysis unit 140 may treat an object identified by the noun "xxx" that appears in association with the above verb as the object of the action. The method by which the document analysis unit 140 determines the description regarding the movement of a specific part of the actor's body is not limited to the example given here.

[0055] The gaze information acquisition unit 150 may acquire gaze information indicating the gaze of the performer via the acquisition unit 71. Furthermore, the gaze information acquisition unit 150 may calculate the gaze information based on the information acquired via the acquisition unit 71.

[0056] The posture information acquisition unit 160 may acquire posture information indicating the posture of the performer via the acquisition unit 71. Furthermore, the posture information acquisition unit 160 may calculate the posture information based on the information acquired via the acquisition unit 71.

[0057] [Processing Procedure of Image Correction Apparatus] Next, a processing procedure of the image correction apparatus according to this embodiment will be described with reference to the flowchart of Fig. 2. Fig. 2 is a flowchart showing processing of the image correction apparatus according to an embodiment of the present disclosure. The processing of the image correction apparatus shown in Fig. 2 may be repeatedly executed at a predetermined cycle.

[0058] In step S101, the acquisition unit 71 acquires an image of the movement of the performer.

[0059] In step S103, the attention part identifying unit 110 identifies an attention part related to the action of the performer that appears in the image.

[0060] In step S105, the predetermined region extraction unit 120 extracts a predetermined region that includes the point of interest therein.

[0061] In step S107, the image correction unit 130 determines the arrangement position of the predetermined area. In other words, the image correction unit 130 corrects the image so that the predetermined area is arranged in the determined arrangement position within the image.

[0062] In step S109, the image correction unit 130 corrects the image by a predetermined correction process.

[0063] In step S111, the output unit 400 outputs the corrected image.

[0064] As described above in detail, the image correction method and image correction device according to this embodiment process an image of a performer's actions captured at a wider angle of view than the predetermined angle of view of a human field of view by identifying a point of interest related to the action that appears in the image, extracting a predetermined region that includes the point of interest, correcting the image by performing a correction process that converts the predetermined region into a simply connected region that corresponds to a narrower angle of view than the predetermined angle of view, and outputting the corrected image.

[0065] As a result, even when a trainee uses images captured with a wider angle of view than the human field of view to learn a movement, it is possible to present the trainee with training images that do not hinder the trainee's understanding of the movement. In particular, since distortion correction processing can be performed to include a specific region without interruption, the burden on the trainee of searching for the specific region in the image can be reduced, improving ease of viewing. Furthermore, the trainee can recognize points that require attention in the movement of the performer, making it easier to learn the performer's movement.

[0066] Furthermore, the image correction method and image correction device according to this embodiment may separate a predetermined region from the boundary of the image through the correction process. This allows for distortion correction processing that seamlessly includes the predetermined region. As a result, the burden on the trainee of searching for the predetermined region in the image is reduced, improving ease of viewing.

[0067] Furthermore, the image correction method and image correction device according to this embodiment may place a predetermined area at the center of the corrected image through the correction process. This allows for distortion correction processing that seamlessly includes the predetermined area. As a result, the burden on the trainee of searching for the predetermined area in the image is reduced, improving ease of viewing.

[0068] Furthermore, in the image correction method and image correction device according to this embodiment, when the image is a panoramic image, the correction process may place a predetermined area in a front position of the corrected panoramic image. When the image is a panoramic image, the image of an object placed in a front position of the panoramic image is displayed with less distortion and in an easier-to-understand manner than an image of an object placed in a position other than a front position of the panoramic image. In this way, by placing the predetermined area in a front position of the panoramic image, the trainee is less likely to be hindered from understanding the point of interest reflected in the predetermined area.

[0069] Furthermore, the image correction method and image correction device according to the present embodiment may identify points of interest by referring to a document describing procedures for a movement. This allows the points of interest associated with the movement to be identified with high accuracy. As a result, the trainee can recognize points of interest in the movement of the performer, making it easier to learn the movement.

[0070] The image correction method and image correction device according to this embodiment may identify points of interest based on nouns or verbs contained in a document. This allows points of interest to be identified taking into account part-of-speech information, such as nouns or verbs, contained in a document describing procedures related to an operation, as well as other contextual information. Furthermore, proper nouns contained in the document and both the source and destination of a task transition, taking into account the context of the document, can be extracted and set as points of interest.

[0071] Furthermore, the image correction method and image correction device according to this embodiment may determine whether a document contains a description regarding the direction of the performer's gaze. If it is determined that the document contains a description regarding the direction of the performer's gaze, the image may be used to extract the performer's gaze direction and identify an object located in the performer's gaze direction as a point of interest. This allows the trainee to be presented with points of interest based on the performer's gaze direction.

[0072] Furthermore, the image correction method and image correction device according to this embodiment may superimpose a graphic indicating a point of interest on the corrected image when it is determined that the document contains a description regarding the direction of the performer's gaze. This allows the trainee to be presented with points of interest based on the direction of the performer's gaze. Furthermore, it is possible to visualize on the video what the performer is focusing on when performing the action, or whether the performer is working without paying particular attention. For example, it is possible to present the trainee with points of interest to become proficient in the task.

[0073] Furthermore, the image correction method and image correction device according to this embodiment may determine whether a document contains a description of the movement of a specific body part of the performer. If it is determined that the document contains a description of the movement of a specific body part of the performer, the specific body part may be extracted based on the image, and the area in which the specific body part moves may be identified as a focus point. This allows the movement of the specific body part of the performer to be presented to the trainee.

[0074] Furthermore, the image correction method and image correction device according to this embodiment may superimpose a graphic indicating the area in which the specific body part moves on the corrected image when it is determined that the document contains a description of the movement of a specific body part of the performer. This allows the trainee to be shown the movement of the specific body part of the performer. Furthermore, by presenting the area in which the specific body part of the performer moves, the range of the performer's movement can be visualized on the image. For example, it is possible to present the trainee with body movements that they should pay attention to in order to become proficient at a task.

[0075] Furthermore, the image correction method and image correction device according to this embodiment may acquire gaze information indicating the gaze of the performer via an acquisition unit, and correct the point of interest based on the gaze information. This allows the point of interest to be identified with high accuracy. For example, the performer may not necessarily be looking at the target object of the action when performing the action. In such cases, the performer may perform the action while looking at an instrument that displays an indicator related to the action. In such cases, instead of the point of interest identified from a work procedure manual or the like, the object located in the performer's line of sight is identified as the point of interest. This allows the point of interest to be identified with high accuracy.

[0076] Furthermore, the image correction method and image correction device according to this embodiment may acquire posture information indicating the posture of the performer via an acquisition unit, and correct the point of interest based on the posture information. This allows the point of interest to be identified with high accuracy. For example, there are cases where the performer does not necessarily look at the target object of the action when performing the action. In such cases, the performer may understand the situation of the action based on their own posture. In such cases, instead of the point of interest identified from a work procedure manual or the like, a specific part of the performer's body identified by posture information indicating the posture of the performer is identified as the point of interest. This allows the point of interest to be identified with high accuracy.

[0077] Furthermore, the image correction method and image correction device according to this embodiment may extract a predetermined region that includes all of the points of interest when multiple points of interest are identified. This allows the trainee watching the video to watch the video without assuming that a noteworthy event will occur on the screen or outside of the field of view. As a result, the trainee can focus their attention on the video and recognize the movement. In addition, the positional relationship between multiple points of interest can be grasped.

[0078] Furthermore, when a movement includes multiple different steps, the image correction method and image correction device according to this embodiment may identify a point of interest for each step. This allows a series of points of interest included in the movement to be presented. As a result, the trainee can recognize points of interest in the movement of the performer, making it easier to learn the movement of the performer.

[0079] Furthermore, the image correction method and image correction device according to the present embodiment may specify the target of the movement as a focus point, thereby enabling the trainee to recognize the target of the movement and facilitate learning the movement of the performer.

[0080] Furthermore, the image correction method and image correction device according to the present embodiment may specify the source and destination of a transition as points of interest when the target of a movement transitions, thereby enabling the trainee to recognize the source and destination of the movement, making it easier to learn the movement of the performer.

[0081] Each of the functions described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), or circuit components arranged to perform the described functions.

[0082] Although the contents of the present disclosure have been described above based on the embodiments, the present disclosure is not limited to these descriptions, and various modifications and improvements are possible, which will be apparent to those skilled in the art. The descriptions and drawings that form part of this disclosure should not be understood as limiting the present disclosure. Various alternative embodiments, examples, and operating techniques will be apparent to those skilled in the art from this disclosure.

[0083] Of course, the present disclosure includes various embodiments not described herein. Therefore, the technical scope of the present disclosure is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description.

[0084] 71 Acquisition unit 73 Database 100 Controller 110 Part of interest identification unit 120 Predetermined area extraction unit 130 Image correction unit 140 Document analysis unit 150 Eye direction information acquisition unit 160 Posture information acquisition unit 400 Output unit

Claims

1. A video correction method for controlling a controller that processes an image obtained by photographing an operator's movement at an angle of view wider than a predetermined angle of view of a person's field of view, wherein the controller: identifies a point of interest related to the movement that appears in the image; extracts a predetermined region that includes the point of interest therein; corrects the image by a correction process of making the predetermined region a singly-connected region corresponding to an angle of view narrower than the predetermined angle of view; and outputs the corrected image.

2. The video correction method according to claim 1, wherein the controller separates the predetermined region from a boundary of the image by the correction process.

3. The video correction method according to claim 1 or 2, wherein the controller arranges the predetermined region at a central position of the corrected image by the correction process.

4. The video correction method according to claim 1, wherein when the image is a panoramic image, the controller arranges the predetermined region at a front position of the corrected panoramic image by the correction process.

5. The video correction method according to any one of claims 1 to 4, wherein the controller identifies the point of interest by referring to a document describing a procedure related to the movement.

6. The video correction method according to claim 5, wherein the controller identifies the point of interest based on a noun or a verb included in the document.

7. The video correction method according to claim 5 or 6, wherein the controller: determines whether the document includes a description regarding the direction of the operator's line of sight; when it is determined that the document includes a description regarding the direction of the operator's line of sight, extracts the direction of the operator's line of sight based on the image; and identifies an object located at the tip of the direction of the operator's line of sight as the point of interest.

8. The video correction method according to claim 7, wherein when it is determined that the document includes a description regarding the direction of the operator's line of sight, the controller superimposes a figure indicating the point of interest on the corrected image.

9. The controller determines whether the document includes a description regarding the movement of a specific part of the operator's body. When it is determined that the document includes a description regarding the movement of a specific part of the operator's body, the specific part is extracted based on the image, and the area where the specific part moves is specified as the target area. The video correction method according to any one of claims 5 to 8, characterized in that.

10. The controller, when it is determined that the document includes a description regarding the movement of a specific part of the operator's body, superimposes a figure indicating the area where the specific part moves on the corrected image. The video correction method according to claim 9, characterized in that.

11. The controller acquires line-of-sight information indicating the line of sight of the operator via an acquisition unit connected to the controller, and corrects the target area based on the line-of-sight information. The video correction method according to any one of claims 1 to 10, characterized in that.

12. The controller acquires posture information indicating the posture of the operator via an acquisition unit connected to the controller, and corrects the target area based on the posture information. The video correction method according to any one of claims 1 to 11, characterized in that.

13. The controller, when a plurality of the target areas are specified, extracts the predetermined area so as to include all the target areas. The video correction method according to any one of claims 1 to 12, characterized in that.

14. The controller, when a plurality of different processes are included in the operation, specifies the target area for each process. The video correction method according to any one of claims 1 to 13, characterized in that.

15. The controller specifies the object of the operation as the target area. The video correction method according to any one of claims 1 to 14, characterized in that.

16. The controller, when the object of the operation transitions, specifies the object before the transition and the object after the transition as the target area. The video correction method according to any one of claims 1 to 15, characterized in that.

17. A video correction device comprising a controller that processes an image obtained by photographing an action of an operator with an angle of view wider than a predetermined angle of view of a human visual field, wherein the controller: identifies a point of interest related to the action that appears in the image; extracts a predetermined region that includes the point of interest therein; corrects the image by a correction process of making the predetermined region a singly connected region corresponding to an angle of view narrower than the predetermined angle of view; and outputs the corrected image. A video correction device characterized by the above.

Images