Design support device and design support method

Abstract

To provide a technology for notifying a user of knowledge of an expert in a short time without requiring labor.SOLUTION: A design support device supports design work by using the line of sight of a designer to an image related to the design work displayed in a display device. The design support device includes a standard knowledge database for storing an image of a region to be taken notice extracted from the line of sight of a first designer to the image related to the design work, a line of sight analysis part for performing line of sight analysis by referring a region extracted from a line of sight of a second designer to the image related to the design work to the region to be taken notice stored in the standard knowledge database, and an update part for excluding the image of a corresponding to the region from the standard knowledge database if the second designer does not look at the region to be taken notice as a result of the line of sight analysis.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a design support apparatus that supports design work by presenting the knowledge of designers, and a design support method.

Background Art

[0002] When carrying out operations in the manufacturing industry, the knowledge possessed by designers is effective. For example, in design work, due to responses to customer requirements or requirements from the manufacturing and maintenance sites, the design of past products may be partially changed. At this time, accurately judging the content of the design change and the scope of influence of the design change is important for maintaining the quality of the product. Design changes are relatively easy for designers who have been engaged in the design work of the target product for many years because they understand the meaning and basis of the design.

[0003] However, for designers with little such experience, it takes a lot of working time to confirm the basis of the design with knowledgeable people and to check past documents, so it is very difficult to make a design change in a short period of time. Here, a skilled designer is distinguished from the perspective of the number of years of engagement in a specific operation and the presence or absence of skill qualifications, and is a person who has excellent knowledge and skills regarding a specific operation.

[0004] In design work, there may be operations such as checking drawings and analyzing the results of analysis such as the finite element method. The operation of checking drawings involves visually inspecting the drawings. The operation of analyzing the results of the finite element method involves, for example, visually checking an image in which the flow of a fluid is visualized. Designers with rich experience in design and analysis operations can execute the operations in a short time without missing the places to be noted in the drawings and analysis results. However, for designers with little experience, there is a possibility of overlooking the places to be checked or missing the characteristic points necessary for interpreting the phenomenon from the analysis results.

[0005] In this way, designers consciously or unconsciously utilize the knowledge they have gained through years of work. Some of the knowledge they consciously utilize is documented and shared as explicit knowledge, while some of the knowledge they unconsciously utilize remains tacit knowledge and is not shared. Explicit knowledge is knowledge that is put into words, while tacit knowledge is knowledge that is not put into words. Explicit knowledge can be referenced by other designers if it is documented, but referencing it requires time to interpret the document. Furthermore, if the amount of knowledge is vast, it takes time to find the desired knowledge. On the other hand, tacit knowledge is possessed only by the designer and is difficult to share.

[0006] Traditionally, designers' knowledge was passed down through on-the-job training (OJT). OJT involves, for example, experienced and inexperienced individuals working together on the same product project over time, with the experienced providing advice to the inexperienced. While tacit knowledge is generally considered difficult to pass on, it was transmitted over time by working on the same project. However, in recent years, such opportunities have decreased due to shorter product cycles and labor shortages. This presents a challenge: if the knowledge of experienced designers is not passed on, and inexperienced designers make design errors, rework can delay delivery dates, and if design errors are discovered after product shipment, it can lead to product recalls.

[0007] As described above, in order to improve the efficiency of design work and enhance product quality, it is necessary to formalize the knowledge of experts, share it with non-experts, and make it usable by non-experts.

[0008] Therefore, technologies are being developed that measure users' gaze and movements, analyze the results to extract the knowledge of experts, create a knowledge database, and present it to inexperienced users. Regarding gaze, technologies are being developed not only in manufacturing but also in other industries to extract the gaze area on screens and images from the user's gaze, and to present the gaze areas of experts to inexperienced users.

[0009] Patent documents 1 and 2 describe technologies for identifying and utilizing areas of focus based on a user's gaze. Patent document 1 discloses a technology for highlighting images of products that a user has focused on when displaying products on a vending machine or the like using a display, based on the user's gaze. Patent document 2 discloses a technology for medical images such as CT scans and X-rays, in which images of areas of focus, including the object of focus detected from the gaze of an expert, are stored in a knowledge database, the area of ​​focus is estimated from a new image using the knowledge database, and it is determined whether or not the user missed the object of focus based on the user's gaze. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Publication No. 2012-022589 [Patent Document 2] WO201MA28237 No. [Overview of the project] [Problems that the invention aims to solve]

[0011] Patent Document 1 envisions the use of a vending machine and identifies the area the user is fixated on based on their gaze. It then assumes that the user is interested in the image contained within the fixated area and enlarges the image or presents related information. While it identifies and remembers the area the user is fixated on based on their gaze, it does not remember the image of the fixated area. Therefore, it is not sufficient as a means of building a knowledge database by remembering important locations and presenting it to other users.

[0012] Patent Document 2 describes a method for acquiring images of a region of interest in medical images, including the object of interest that a skilled user has focused on. After confirming that the image content is important by interviewing the user, the images are stored in a knowledge database. The stored images are used as input data for a machine learning model to detect regions of interest in new medical images. The machine learning model estimates the region of interest in the new medical image and presents it to the user. New images of objects of interest are added to the knowledge database, but unnecessary images are not deleted to update the knowledge database. Therefore, if there are multiple types of objects of interest, and their priority changes depending on the task, causing some to cease being objects of interest, unnecessary images may remain in the knowledge database, potentially leading to the detection of incorrect regions of interest. If the object of interest changes, rebuilding the knowledge database requires time-consuming processes such as measuring the user's gaze and conducting interviews.

[0013] In design work, for example, when interpreting phenomena from images of fluid analysis results, if the work changes from designing turbomachinery where pressure loss is the focus to designing blades where vortices caused by separation are the focus, turbomachinery requires images visualizing the distribution of pressure and velocity, but blades do not require these images; instead, images of vortices are needed. Thus, when interpreting phenomena from images of fluid analysis results, it is necessary to update the knowledge database to match the target product.

[0014] This invention has been made in view of the above-mentioned problems, and aims to provide a design support device and design support method that automatically updates a knowledge database when the object of focus changes, in a technology that displays an area of ​​interest on the user's work execution screen. [Means for solving the problem]

[0015] Based on the above, the present invention relates to "design work displayed on a display device." two dimensional A design support device that assists design work using the designer's gaze on an image, The aforementioned Regarding design work two dimensional The primary designer's perspective on the image Based on the above, the area that the first designer focused on in the two-dimensional image is determined from the speed of the first designer's gaze movement and the time spent within a certain range. The image of the first area Get Reference knowledge database And , The second display on the second display device Regarding the design work Second 2D The line of sight of the second designer with respect to the image Based on the above, the region that the second designer focused on in the second two-dimensional image is determined from the speed of the second designer's gaze movement and the time spent within a certain range. Second area Image to The eye-tracking analysis unit acquires data, and on the second two-dimensional image, The first area stored in the reference knowledge database The gaze analysis unit comprises a gaze area estimation unit that estimates a region similar to the second region as a region to be gazed upon, and the gaze analysis unit compares the second region and the region to be gazed upon. Refer to Whether the second designer observed the area that should be observed, or whether he observed the area that should not be observed. Gaze analysis death , The aforementioned As a result of the gaze analysis, if the second designer does not Must keep a close eye on area Keep a close eye on it If not Similar to the area that requires attention. do The first of the above A design support device characterized by comprising an update unit that excludes the image of the area from the reference knowledge database.]] It is what was made.

[0016] Also in the present invention, " A design support device and a design support method that support design work using the gaze of a designer on a two-dimensional image relating to design work displayed on a display device, wherein the design support device comprises a gaze analysis unit, a gaze area estimation unit, and an update unit, wherein the gaze analysis unit acquires an image of a first area, which is the area that the first designer gazed on, based on the gaze of a first designer on the two-dimensional image relating to the design work, from the movement speed of the gaze of the first designer and the time spent in a certain range on the two-dimensional image, and stores it in a reference knowledge database, and based on the gaze of a second designer on a second two-dimensional image relating to a second design work displayed on a second display device, the gaze of the second designer A design support method characterized by: acquiring an image of a second region, which is the region that the second designer has been fixated on, based on the speed of movement of a line and the time it stays within a certain range; the fixation region estimation unit estimates a region on the second two-dimensional image that is similar to the first region stored in the reference knowledge database as a region to be fixed on; the gaze analysis unit refers to the second region and the region to be fixed on and performs gaze analysis to determine whether the second designer has been fixated on the region to be fixed on or on a region that should not be fixed on; and if, as a result of the gaze analysis, the second designer has not been fixated on the region to be fixed on, the update unit removes an image of the first region that is similar to the region to be fixed on from the reference knowledge database. " It is what was made.

Effects of the Invention

[0017] In the technology of presenting a target area on the user's business execution screen, the knowledge database can be automatically updated when the target of attention changes.

Brief Description of the Drawings

[0018] [Figure 1] A diagram showing the organization of the database generation stage. [Figure 2] A diagram showing a configuration example in the creation process of the design support device according to Embodiment 1 of the present invention. [Figure 3] A diagram showing a structural example of expert knowledge stored in the knowledge database. [Figure 4] A diagram showing an example of expert knowledge displayed on the business execution screen. [Figure 5] A diagram showing a configuration example in the update process of the design support device according to Embodiment 1 of the present invention. [Figure 6] A diagram showing an example of a screen for notifying the user that they have gazed at an area other than the area estimated from the knowledge database. [Figure 7] A flowchart for updating a knowledge database according to Embodiment 1 of the present invention. [Figure 8] This figure shows an example configuration of a design support device according to Embodiment 2 of the present invention. [Figure 9] A flowchart for updating a knowledge database according to Embodiment 2 of the present invention. [Modes for carrying out the invention]

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0020] In the following, Example 1 will describe the assumed usage scenario, followed by a description of the apparatus and method related to the embodiment of the present invention, and Example 2 will describe a modified embodiment. [Examples]

[0021] In the preceding section of Example 1, we will clarify an example of the usage scenario assumed in the present invention.

[0022] Reducing the effort required to create a knowledge database for designers will enable the construction of such a database in a shorter period, leading to more efficient knowledge transfer among designers that does not rely on on-the-job training (OJT).

[0023] Therefore, assuming that the areas to be observed are common or similar for the same type of work, a standard knowledge database created in advance from the same type of work is updated to create a new knowledge database. Specifically, the areas to be observed based on the knowledge database created by the first group of designers are measured to see whether the second group of designers, whose work is different, has focused on them, and the knowledge database is updated to suit the second group of designers. This allows the knowledge database used by the second group of designers to be created in a short time. Here, "focusing" means that the gaze remains fixed on a specific area without moving.

[0024] Figure 1 is a diagram illustrating the stages of database creation. The initial creation stage is defined as the stage in which the first group of designers, MA1 and MA2, create the knowledge database DBα, and the deployment and application stage is defined as the stage in which the second group of designers, MB1 and MB2, who perform similar tasks, modify and utilize the knowledge database DBβ. In Figure 1, the general processing content in the initial creation stage is shown as the creation process 10A, and the general processing content of the design support device in the deployment and application stage is shown as the update process 10B1 and the deployment process 10B2.

[0025] In this example, we will explain that in the initial creation phase, in creation process 10A, the first group of designers MA1 and MA2 belonging to Company A create the knowledge database DBα. In the deployment and application phase, in update process 10B1, the created knowledge database DBα is deployed to Company B. Furthermore, in deployment process 10B2, the second group of designers MB1 and MB2 belonging to Company B modify it into a knowledge database DBβ with content suitable for their own use. When the operations of Company A and Company B are similar or related, building the knowledge database DBβ at Company B by modifying a similar device is expected to be more efficient and completed sooner than building it from scratch. Needless to say, the same can be done even within the same company.

[0026] The process 10A for creating the standard knowledge database DBα at Company A is performed using the standard knowledge database DBα, the gaze analysis unit 7A of skilled MA1 and unskilled MA2 belonging to the first group of designers, the gaze measurement device 6A, and the monitor 90A.

[0027] The eye-tracking device 6A measures the gazes of expert MA1 and unskilled MA2 in the fluid analysis result images. When the eye-tracking device 6A takes measurements, it may use, for example, the work execution screen displayed on monitor 90A. In the gaze analysis unit 7A of expert MA1 and unskilled MA2, experts in fluid analysis and eye-tracking extract images corresponding to the area of ​​interest, including the object of interest, based on the analysis of the eye-tracking results and interviews with expert MA1 and unskilled MA2, and store them in the reference knowledge database DBα.

[0028] The update process 10B1 for the standard knowledge database DBα at Company B is performed using an eye-tracking device 6B1, an analysis unit 7B1 for analyzing the eye-tracking data of the expert MB1, a database update unit (knowledge presentation control unit) 4B1, and a monitor 90B1.

[0029] The expert's gaze data analysis unit 7B1 estimates the gaze area of ​​expert MB1 in the work image displayed on monitor 90B1 using the reference knowledge database DBα. The gaze measurement unit 6B1 measures the gaze of expert MB1 from Company B in the work image. The database update unit 4B1 updates the reference knowledge database DBα based on the estimation results of the gaze area in the work image and the gaze measurement results, and creates the knowledge database DBβ.

[0030] The process 10B2 for using the knowledge database DBβ at Company B is executed using the eye-tracking device 6B2, the eye-tracking data analysis unit 7B2, the knowledge database DBβ, the input image acquisition unit 3B, the knowledge presentation control unit 4B2, the knowledge output unit 5B, and the monitor 90B2.

[0031] The input image acquisition unit 3B acquires an image (e.g., a screenshot) of the screen displayed on the business execution screen shown on the monitor 90B2. The acquired image is sent to the knowledge presentation control unit 4B2. The knowledge presentation control unit 4B2 uses the knowledge database DBβ to estimate the areas of focus in the input image that the user (MB2, an unskilled employee of Company B) should look at.

[0032] The eye-tracking device 6B2 measures the position that the unskilled worker MB2 is fixating on during work on the work execution screen. The obtained results are transmitted to the eye-tracking data analysis unit 7B2. The eye-tracking device 6B2 continuously measures the gaze of the unskilled worker MB2. The knowledge output unit 5B compares the location that the unskilled worker MB2 is fixating on with the location that should be looked at, output from the knowledge presentation control unit, and displays the locations that the unskilled worker MB2 is not fixating on on the unskilled worker MB2's work execution screen from the knowledge output unit 5B.

[0033] In cases where the tasks at Company A and Company B involve searching for vortices from fluid analysis results, and are therefore common or similar, Company A's standard knowledge database DBα can be updated with the line-of-sight measurement results of Company B's experts, allowing Company B to quickly create its own knowledge database DBβ and easily utilize the expertise of its experts in its operations.

[0034] Figure 2 is a block diagram showing an example configuration of a design support device 10 according to Embodiment 1 of the present invention. It shows a common device configuration that can perform the processing in each process shown in Figure 1. The design support device 10 comprises a knowledge database DB, a work execution screen 2, an input image acquisition unit 3, a knowledge presentation control unit 4, a knowledge output unit 5, an eye-tracking device 6, an eye-tracking data analysis unit 7, and a monitor 90.

[0035] Figure 2 illustrates the design support device 10 shown in Figure 2, intended to be used in the creation process 10A of the reference knowledge database DBα at Company A. The knowledge database DB (reference knowledge database DBα in the creation process 10A) is a database that stores a large number of images of points of interest to be viewed. Separately, images of important locations are extracted and stored by analyzing and interviewing the gaze of an expert MA1 and an inexperienced MA2.

[0036] Task execution screen 2 is the screen displayed on monitor 90 where the designer performs the work. The input image acquisition unit 3 acquires an image (e.g., a screenshot) of the screen displayed on task execution screen 2. The acquired image is transmitted to the knowledge presentation control unit 4.

[0037] The knowledge presentation control unit 4 uses the knowledge database DB to estimate the areas of interest that the user should focus on in the input image. From the image in the business execution screen, it uses, for example, machine learning-based object detection technology to estimate areas similar to images stored in the knowledge database DB. Object detection is a technology that identifies the location and number of specified objects in an image, and is used in applications such as facial recognition in smartphones and pedestrian detection in autonomous driving.

[0038] The eye-tracking device 6 measures the user's gaze while they are working on the work execution screen 2. The eye-tracking device 6 can use methods such as the corneal reflection method, which involves capturing a reflection pattern generated on the cornea of ​​the eyeball by a near-infrared LED (Light Emitting Diode) with a camera and calculating the direction of the eyeball from the point of light reflection.

[0039] The obtained results are transmitted to the eye-tracking data analysis unit 7. The eye-tracking data analysis unit 7 calculates the locations the user is fixated on based on factors such as the speed of eye movement and the time spent in a certain area. The eye-tracking device 6 continuously measures the user's gaze. The knowledge output unit 5 compares the locations the user is fixated on with the locations to be viewed output from the knowledge presentation control unit, and displays the locations the user is not fixated on on the user's work execution screen 2.

[0040] Thus, by analyzing the gaze of expert MA1 and inexperienced MA2, the image regions that users focused on are stored in the knowledge database DB as points of interest to be viewed, thereby creating the standard knowledge database DBα for Company A.

[0041] Figure 3 shows an example of an image stored in the knowledge database DB (the image region the user focused on, the point of interest to be observed) when the task is to find vortices from fluid analysis results. Hereafter, the details of the present invention will be explained using the example of presenting the areas that the expert MA1 focuses on in the task of finding vortices from fluid analysis results to users who will use the device at a later date, as part of the expert MA1's knowledge.

[0042] The knowledge database DB stores multiple images of locations to look at, extracted based on the analysis results of the gaze patterns of expert MA1 and unskilled MA2, as illustrated in Figure 3, 1 to 6. In this example, the shape of the images may be an ellipse corresponding to the shape of the vortex, or a rectangular shape containing a vortex. Furthermore, the locations to look at may not be images, but other parameters that can identify the locations to look at, such as an array of numbers equivalent to an image.

[0043] Figure 4 shows an example of the results of estimating and visualizing the areas to be observed using images from the knowledge database DB in an input image acquired from the work execution screen 2. Figure 4 is a visualization of streamlines, one of the results of fluid analysis, in which multiple vortices are generated. The areas to be observed (regions) 301, extracted and recorded when expert MA1 pauses the image, are visualized with rectangular frames, similar to images stored in the knowledge database DB, and displayed on the monitor 90. This allows inexperienced MA2 to learn where to look when searching for vortices and to acquire the knowledge of expert MA1.

[0044] Figure 5 illustrates the design support device 10 shown in Figure 2, intended to be used in the update process 10B1 of the reference knowledge database DBα at Company B. Figure 5 is a block diagram primarily showing the functions of the knowledge presentation control unit 4 in the update process 10B1.

[0045] This section explains how to update and create Company B's standard knowledge database DBβ by deleting, replacing, or adding images of the locations to be viewed, when deploying Company A's standard knowledge database DBα for use at Company B. This allows for the rapid preparation of knowledge database DBβ even when the focus of attention changes.

[0046] In this case, the knowledge presentation control unit 4 functions as the knowledge database update unit 4B1, and the design support device 10B1 includes the processing functions of a reference knowledge database DBα, an input image acquisition unit 3, a gaze area estimation unit 403 in the work image, a gaze measurement device 6, a gaze analysis unit 7, and a database update unit 406.

[0047] The reference knowledge database DBα is a database that stores images of points of interest that have been created in advance. However, the images of points of interest stored in the reference knowledge database DBα at this time are data extracted from the perspective of Company A, and are not necessarily images from the perspective suitable for use by Company B.

[0048] The input image acquisition unit 3 acquires an image (e.g., a screenshot) of the screen displayed on the business execution screen 2. The gaze area estimation unit 403 in the business image takes an image from the reference knowledge database DBα as input and uses techniques such as object detection to estimate locations in the image that are similar to the input image. The estimated locations become the areas to be viewed in the business image.

[0049] The eye-tracking unit 6 measures the user's gaze position on the work execution screen during work. The eye-tracking analysis unit 7 analyzes whether the user gazed at the area they should have been looking at. It also analyzes whether the user gazed at an area they should not have been looking at. The database DB update unit 406 updates the image of the reference knowledge database DBα in accordance with the presence or absence of gaze output from the eye-tracking analysis unit 7.

[0050] If areas that should have been monitored were not monitored, even if those areas were important in the business that created the standard knowledge database DBα, they will be considered unimportant in the business of Company B's users, and the images corresponding to those areas will be excluded from the standard knowledge database DBα. Here, exclusion means deleting from the standard knowledge database DBα, setting a flag indicating that the data is not used without deleting it, or moving the data to another database as excluded data. In any case, it means that the data will be treated as data that will not be considered in Company B's use.

[0051] If the user is focusing on a specific area, the image in the reference knowledge database DBα may be retained as is, or it may be replaced with an image extracted from the area of ​​focus in the work image. If the user is focusing on an area that is not a specific area, the image corresponding to that area should be added to the reference knowledge database DBα. Alternatively, if the user is focusing on an area that is not a specific area, a message may be displayed on the screen indicating that the user is focusing on an area not present in the reference knowledge database DBα. The user can then confirm whether it is the area of ​​focus, and if it is, they can specify the area of ​​the image to be added to the reference knowledge database DBα.

[0052] Thus, by analyzing the gaze of MB1, a skilled user at Company B, the image regions that MB1 focused on are newly stored in the knowledge database DBα as points of interest to be viewed, and the image regions that MB1 did not focus on are removed from the knowledge database DBα to create a standard knowledge database DBβ suitable for use at Company B.

[0053] Figure 6 shows an example of information displayed on the user's work execution screen when the user focuses on an area they should not be focusing on. Areas 501 that the user (expert MB1) focuses on, other than those estimated from the reference knowledge database DBα, are indicated by rectangles or circles. The comment 502 to the user states that the user focused on areas other than those estimated from the reference knowledge database DBα, and that the size of the rectangles or circles should be adjusted to match the important areas.

[0054] Figure 7 is a flowchart for updating the knowledge database DBα based on the block diagram in Figure 5. It is effective to use the gaze of expert MB1 when updating the knowledge database DBα.

[0055] In the first processing step S601 of the process shown in Figure 7, the business image displayed on the business execution screen is read. In processing step S602, the areas to be focused on in the business image are estimated using a pre-created reference knowledge database DBα. At this time, the areas to be focused on are not displayed on the business image.

[0056] In processing step S603, the gaze of the skilled worker MB1 during work is measured. In processing step S604, it is determined whether user MB1 did not gaze at the areas that should have been gazed upon. If there were areas that were not gazed upon, in processing step S605, the images of the areas that the user did not gaze at among the areas that should have been gazed upon are removed from the reference knowledge database DBα.

[0057] If user MB1 has focused on all the areas to be focused on, the process proceeds to step S606, where the image of the knowledge database DBα corresponding to the areas to be focused on is retained, or replaced with an image of the business image corresponding to the areas to be focused on.

[0058] If the business image used when creating the knowledge database DBα differs from the business image used when updating the knowledge database DBα, swapping the images will improve the accuracy of the gaze region estimation by the updated knowledge database DBβ.

[0059] In processing step S607, it is determined whether user MB1 has focused on an area that is not part of the area that should be focused on. If user MB1 has focused on an area that is not part of the area that should be focused on, in processing step S607, an image of the area that user MB1 focused on is extracted and added to the reference knowledge database DBα.

[0060] In this way, the reference knowledge database DBα is updated by deleting images of areas that the expert MB1 did not focus on among the areas that should be focused on, retaining or replacing images of areas that the expert MB1 focused on, and adding images of areas that were focused on but not the areas that should be focused on, thereby creating a knowledge database DBβ suitable for use by Company B.

[0061] Furthermore, when configuring the apparatus of the present invention, these devices do not necessarily have to be installed in the same location; for example, some of their functions can be configured on the cloud. [Examples]

[0062] Example 2 describes a method for efficiently and accurately performing the update process 10B1 of the knowledge database DBα at Company B.

[0063] Figure 8 is a block diagram showing the functions of the knowledge presentation control unit 4 (4B1) in the update process 10B1. The design support device 10 at this time includes a reference knowledge database DBβ, an input image acquisition unit 3, a gaze area estimation unit 403 in the work image, a gaze measurement unit 6, a gaze analysis unit 7, a gaze count comparison unit 706, and a database DB update unit 406.

[0064] The reference knowledge database DBα is a database that stores images of objects of interest that should be viewed, which have been created in advance by Company A. The input image acquisition unit 3 acquires images of the screen displayed on the business execution screen (e.g., screenshots). The gaze area estimation unit 403 in the business image takes an image from the reference knowledge database DBα as input and uses object detection technology to estimate locations in the image that are similar to the input image. The estimated locations become the places to be viewed in the business image.

[0065] The eye-tracking device 6 measures the location on the work execution screen that the user (expert MB1) is fixated on during work. The eye-tracking analysis unit 7 analyzes whether the user is fixated on the location they should be fixated on, and also analyzes whether they are fixated on a location that they should not be fixated on.

[0066] The gaze count comparison unit 706 stores the results of measuring the gaze of multiple skilled MB1s, counts the areas that were gazed upon and the number of times they were gazed upon, and selects images to add or delete according to the count. The database DB update unit 406 updates the images in the reference knowledge database DBβ according to the presence or absence and frequency of gaze from the gaze count comparison unit 706. By measuring the gaze of multiple skilled MB1s in work images, comparing the number of times each area was gazed upon, and updating the reference knowledge database DBβ according to the magnitude of the comparison, the database DBβ can be updated to be unbiased towards the knowledge of any particular skilled person.

[0067] Figure 9 is a flowchart for updating the knowledge database DBα based on the block diagram in Figure 8. Note that using the gaze of experts is effective when updating the knowledge database DBβ. In processing step S801, the work image displayed on the work execution screen is read. In processing step S802, the areas to be focused on in the work image are estimated using the pre-created reference knowledge database DBα. At this time, the areas to be focused on are not displayed on the work image. In processing step S803, the gaze of multiple experts during their work is measured.

[0068] In processing step S804, the frequency of gaze counts for each area to be focused on is analyzed from the gaze measurement results of multiple skilled individuals. The frequency of gazes on areas other than those to be focused on is also analyzed. In processing step S805, images of areas to be focused on with high frequency are retained, or if the images in the reference knowledge database DBα differ from the work images, the images of areas to be focused on in the work images are replaced. In processing step S806, images of areas to be focused on with low gaze frequency are deleted from the reference knowledge database DBα. In processing step S807, images of areas other than those to be focused on with high gaze frequency are added.

[0069] In this way, the reference knowledge database DBα is updated by replacing, deleting, and adding images in the reference knowledge database DB based on the gaze measurement results of multiple experts, according to the frequency of gaze. [Explanation of Symbols]

[0070] 10: Design support device, DB, DBα, DBβ: Reference knowledge database, 2: Business execution screen, 3: Input image acquisition unit, 4: Knowledge presentation control unit, 5: Knowledge output unit, 6: Eye-tracking device, 7: Eye-tracking data analysis unit, 403: Gaze area estimation unit in business images, 406: Database DB update unit, 501: Locations the user gazed at other than those estimated from the reference knowledge database DB, 502: Comments to the user, 706: Gaze count comparison unit, 10A: Construction plan for the reference knowledge database DB at Company A Process, 7A: Eye-tracking analysis unit for skilled and unskilled individuals, 6A: Eye-tracking device, MA1: Skilled individuals, MA2: Unskilled individuals, 10B: Standard knowledge database DB update process at Company B, 6B1: Eye-tracking device, 7B1: Skilled individuals' eye-tracking data analysis unit, 4B1: Database DB update unit, 10C: Knowledge database DB utilization process at Company B, 6B2: Eye-tracking device, 7B2: Eye-tracking data analysis unit, 4B2: Knowledge presentation control unit, 5B: Knowledge output unit, 3B: Input image acquisition unit, MB2: Unskilled individuals

Claims

1. A design support device that assists design work by using the designer's gaze on an image related to the design work displayed on a display device, A gaze analysis unit extracts an image of a first region, which is a partial area that the first designer fixated on, from the speed of movement of the first designer's gaze and the time the designer remained within a certain range on the two-dimensional image, based on the gaze of the first designer displayed on the first display device, and stores it in a reference knowledge database. A gaze analysis unit also extracts an image of a second region, which is a partial area that the second designer fixated on, from the speed of movement of the second designer's gaze and the time the designer remained within a certain range on the second two-dimensional image, based on the gaze of the second designer displayed on the second display device, based on the gaze of the second designer's gaze A gaze region estimation unit estimates a sub-region on the second two-dimensional image that is similar to the first region stored in the reference knowledge database as a region to be gazed upon, Equipped with, The gaze analysis unit refers to the second area and the area to be gazed upon and performs gaze analysis to determine whether the second designer gazed upon the area to be gazed upon or upon an area that should not be gazed upon. A design support device characterized by comprising an update unit that, as a result of the gaze analysis, removes from the reference knowledge database images of the first region that are similar to the region that the second designer did not gaze upon among the regions to be gazed upon.

2. A design support device according to claim 1, The update unit is characterized in that, when the second designer looks at the area to be looked at, it holds an image of the first area in the reference knowledge database that is similar to the area to be looked at, or replaces the image of the area to be looked at by the second designer with an image of the first area in the reference knowledge database that is similar to the area to be looked at.

3. A design support device according to claim 1, The update unit is a design support device characterized in that, when the second designer gazes upon an area other than the area to be gazed upon, the image of the area other than the area to be gazed upon by the second designer is added to the reference knowledge database.

4. A design support device according to claim 1, The update unit is characterized in that, as part of the gaze analysis of multiple second designers, it determines to exclude images of the first region similar to the region to be focused on from the reference knowledge database, according to the frequency with which the multiple second designers observe the region to be focused on.

5. A design support device according to claim 4, The design support device is characterized in that the update unit holds an image of the first region of the reference knowledge database that is similar to the region to be observed frequently, or replaces the image of the region to be observed frequently with an image of the first region of the reference knowledge database that is similar to the region to be observed.

6. A design support device according to claim 4, The update unit is a design support device characterized in that, when the second designer gazes at an area other than the area to be gazed at, images of areas other than the area to be gazed at that have a high frequency of gaze are added to the reference knowledge database.

7. A design support device according to claim 1, The design support device is characterized in that exclusion from the aforementioned reference knowledge database means deleting it from the reference knowledge database, setting a flag indicating it is not in use without deleting it, or moving it to another database as excluded data.

8. A design support method in a design support device that supports design work using the gaze of a designer on a two-dimensional image relating to design work displayed on a display device, The design support device includes a gaze analysis unit, a gaze area estimation unit, and an update unit. The aforementioned gaze analysis unit is Based on the gaze of the first designer on the two-dimensional image relating to the design work, an image of the first region, which is a partial area that the first designer focused on, is acquired from the movement speed of the first designer's gaze and the time spent within a certain range on the two-dimensional image and stored in the reference knowledge database. Also, based on the gaze of the second designer on the second two-dimensional image relating to the second design work displayed on the second display device, an image of the second region, which is a partial area that the second designer focused on, is acquired from the movement speed of the second designer's gaze and the time spent within a certain range on the second two-dimensional image. The aforementioned gaze area estimation unit, On the second two-dimensional image, a sub-region similar to the first region stored in the reference knowledge database is estimated as a region to be focused on. The aforementioned gaze analysis unit is Referencing the second area and the area to be observed, an eye-tracking analysis is performed to determine whether the second designer gazed upon the area to be observed or upon an area that should not be observed. The aforementioned update unit is If, as a result of the gaze analysis, the second designer does not gaze at the area that should be gazed upon, then an image of the first area similar to the area that should be gazed upon is removed from the reference knowledge database. A design support method characterized by the following features.

Images