State diagnosis system, state diagnosis method, and program

Abstract

This state diagnosis system includes: an acquisition means for acquiring an image including a diagnosis target; a specification means for specifying a damaged portion and type of damage of the diagnosis target in the image acquired by the acquisition means by using a trained model that receives input of an image to output a damaged portion and type of damage of a diagnosis target included in the image; and an output means for outputting the image acquired by the acquisition means and the damaged portion and type of damage specified by the specification means in association with each other.

Description

Condition diagnosis system, condition diagnosis method, and program

[0001] The present invention relates to a condition diagnosis system, a condition diagnosis method, and a program.

[0002] Conventionally, various parts are used in machinery, and stable operation of machinery is achieved by properly managing the condition of these parts. The condition of parts in machinery is diagnosed by visual inspection by workers, and after confirming the condition of the parts, replacement or repair is carried out.

[0003] Because such work depends on the skill and experience of the worker, diagnosing the condition of the machinery has not been easy. In response to this, for example, Patent Document 1 discloses a method for acquiring photographed images of external parts of a vehicle and determining damage to the external parts using a part learning model and a state learning model. Patent Document 2 also discloses a method for acquiring images of the tread surface of a tire and diagnosing the degree of uneven tire wear using trained AI.

[0004] Japanese Patent No. 6991519 Japanese Patent Application Laid-Open No. 2023-55602

[0005] Take a bearing device as an example of a mechanical device that can have replaceable parts. Damage to each component of a bearing device can occur in a variety of states. To perform a more appropriate condition diagnosis, other condition information may be required in addition to the component's appearance. For example, an experienced technician can determine whether additional measurements or information should be acquired based on the currently acquired information. Furthermore, an experienced technician may be able to determine whether part replacement or repair is the appropriate course of action in the current condition. However, an inexperienced technician may not be able to fully determine what action to take when diagnosing the condition. In such cases, the method described in Patent Document 1 or the like cannot be used as is.

[0006] In view of the above problems, an object of the present invention is to realize a condition diagnosis method for mechanical devices that can be easily introduced and has higher usability.

[0007] In order to solve the above problems, the present invention has the following configuration: A condition diagnosis system includes: an acquisition means for acquiring an image including a diagnosis target; an identification means for identifying a damaged portion and a type of damage in the image acquired by the acquisition means using a trained model that receives an image as an input and outputs a damaged portion and a type of damage of the diagnosis target contained in the image; and an output means for outputting the image acquired by the acquisition means in association with the damaged portion and the type of damage identified by the identification means.

[0008] Another aspect of the present invention has the following configuration: a condition diagnosis method includes an acquisition step of acquiring an image including a diagnosis target, an identification step of identifying a damaged portion and a type of damage in the image acquired in the acquisition step using a trained model that receives the image as input and outputs a damaged portion and a type of damage of the diagnosis target included in the image, and an output step of outputting the image acquired in the acquisition step in association with the damaged portion and the type of damage identified in the identification step.

[0009] Another aspect of the present invention has the following configuration: That is, a program causes a computer to execute: acquisition means for acquiring an image including a diagnostic object; identification means for identifying a damaged portion and a type of damage in the image acquired by the acquisition means using a trained model that receives an image as input and outputs a damaged portion and a type of damage of the diagnostic object contained in the image; and output means for outputting the image acquired by the acquisition means in association with the damaged portion and the type of damage identified by the identification means.

[0010] The present invention makes it possible to realize a condition diagnosis method for mechanical devices that can be easily introduced and has higher usability.

[0011] FIG. 1 is a schematic diagram showing an example of a system configuration according to an embodiment of the present invention. FIG. 2 is a flowchart of real-time diagnosis processing according to an embodiment of the present invention. FIG. 3 is a flowchart of detailed diagnosis processing according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of the configuration of a UI for image acquisition according to an embodiment of the present invention. FIG. 5 is a diagram showing an example of the configuration of a UI for condition diagnosis according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of content for additional measurement according to an embodiment of the present invention. FIG. 7 is a diagram showing an example of a correspondence table for condition diagnosis according to an embodiment of the present invention. FIG. 8 is a diagram showing an example of a correspondence table for condition diagnosis according to an embodiment of the present invention.

[0012] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the embodiment described below is one embodiment for explaining the present invention and is not intended to be interpreted as limiting the present invention. Furthermore, not all of the configurations described in each embodiment are necessarily essential configurations for solving the problems of the present invention. Furthermore, in each drawing, the same components are assigned the same reference numerals to indicate corresponding relationships.

[0013] In the following description, "learning" or "machine learning" refers to generating a "trained model" by repeatedly performing a learning process using training data and an arbitrary learning algorithm. A trained model is updated as needed as learning progresses using multiple pieces of training data, and its output changes even for the same input. Therefore, a trained model is not limited to a specific state at any point in time. Here, a model used in learning will be referred to as a "trained model," and a learning model that has undergone a certain level of learning will be referred to as a "trained model."

[0014] Specific examples of "learning data" will be described later, but the configuration of the data may be adjusted or changed depending on the learning algorithm used, its purpose, and the type of device to be diagnosed. For example, the learning data may include image data, which will be described later, as well as vibration data, sound data, ultrasound data, and numerical data. The configuration of the learning data and the preprocessing of the learning data may vary depending on the type of learning algorithm and the input and output contents of the learning model.

[0015] Furthermore, the training data may include training data used for training itself, verification data used for verifying a trained model, and test data used for testing a trained model. In the following description, data related to training will be referred to as "training data" when referring to them collectively. Note that it is not intended to clearly classify the training data into training data, verification data, and test data; for example, depending on the training, verification, and testing methods, all training data may also be training data.

[0016] <First embodiment> A first embodiment of the present invention will be described below. In this embodiment, a bearing device will be described as an example of a diagnostic target. However, the present invention can be applied to any device that performs diagnostics for component replacement, repair, etc., as described below.

[0017] [System Configuration] Hereinafter, one aspect of a system to which the technique according to this embodiment can be applied will be described. FIG. 1 is a schematic diagram showing an example of a system configuration according to this embodiment. As shown in FIG. 1, the system according to this embodiment includes a terminal device 100 and a server device 200, which are communicatively connected via a network 300. Note that this embodiment will be described using one terminal device 100 and one server device 200 as an example, but this is not limiting. A configuration in which multiple devices of each type are provided and cooperate to share various processes according to their functions may also be used.

[0018] The terminal device 100 is a device that can be used in the vicinity of the bearing device (not shown) that is the target of diagnosis, and may be, for example, a smartphone, a tablet terminal, a POS terminal, a dedicated device, etc. The terminal device 100 includes a processing unit 101, a storage unit 102, an imaging unit 103, a UI (User Interface) unit 104, and a communication unit 105. The processing unit 101 may be composed of a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a dedicated circuit.

[0019] The storage unit 102 is composed of volatile and non-volatile storage media such as a hard disk drive (HDD), read-only memory (ROM), random access memory (RAM), or flash memory, and is capable of inputting and outputting various information in response to instructions from the processing unit 101. The imaging unit 103 is, for example, a camera, and is capable of acquiring real-time images and still images of the bearing device to be diagnosed. The UI unit 104 displays a UI screen (described later) and accepts operations from the user. The UI unit 104 may be composed of, for example, a touch panel display. The communication unit 105 is a communication interface that controls communication with an external device (e.g., the server device 200). The communication standard used by the communication unit 105 via the network 300 is not particularly limited, and multiple communication standards may be combined.

[0020] The server device 200 is an information processing device for providing functions described below in cooperation with the terminal device 100. The server device 200 may be constructed as an on-premise type in an environment where the terminal device 100 is used, or may be constructed as a cloud type on the Internet. The network 300 may be constructed by combining, for example, a local area network (LAN), a wide area network (WAN), the Internet, etc.

[0021] The server device 200 includes a processing unit 201, a storage unit 202, a communication unit 203, and an input / output unit 204. The processing unit 201 may be configured with a CPU, a GPU, a GPGPU, an MPU, a DSP, an FPGA, or a dedicated circuit. For example, the processing unit 201 of the server device 200 may have a configuration suitable for a learning process for generating a trained model, which will be described later.

[0022] The storage unit 202 is composed of volatile and non-volatile storage media such as a HDD, ROM, and RAM, and is capable of inputting and outputting various types of information in response to instructions from the processing unit 201. The communication unit 203 is a communication interface that controls communication with an external device (e.g., the terminal device 100). The standard of communication by the communication unit 203 via the network 300 is not particularly limited, and multiple communication standards may be combined. The input / output unit 204 is an interface for inputting and outputting various types of data and instructions.

[0023] In the system according to this embodiment, it is assumed that an operator (e.g., a rolling device manager) uses the terminal device 100 to diagnose the condition of the rolling device. The functions described below may be provided by cooperation between the terminal device 100 and the server device 200. Note that the division of processing shown below is an example, and the subject of processing may be changed depending on the application environment, etc. Furthermore, the functions described below may be realized as an application installed on the terminal device 100, or as a web application provided by the server device 200. Alternatively, they may be realized by a combination of these.

[0024] Generally, the learning process for generating a trained model imposes a large processing load. Therefore, the trained model used in this embodiment will be described as being generated by a learning process performed at a predetermined timing on the server device 200 side. At this time, the server device 200 appropriately collects training data and repeats the learning process. The server device 200 stores and manages multiple trained models according to the level of learning and the type of rolling device to be diagnosed. Then, the server device 200 provides the trained model to the terminal device 100 at a predetermined timing, allowing the terminal device 100 to use the trained model during the diagnostic process.

[0025] The trained model according to this embodiment is a trained model that receives an input image containing components constituting a rolling device, and undergoes a training process to identify and output damaged parts and types of damage in the components from the image. The training algorithm here is not particularly limited, but may be, for example, a well-known deep learning method such as R-CNN (Region Convolutional Neural Network), YOLO (You Only Look Once), or SSD (Single Shot MultiBox Detector).

[0026] Furthermore, as the learning data used in the learning process, data in which annotation data indicating damaged portions and their types is associated with image data is used. Note that the learning data may be generated manually or using a predetermined tool. The generation of the learning data itself may be performed using a known method, and a detailed description thereof will be omitted here.

[0027] [Processing Flow] The condition diagnosis process according to this embodiment will be described below. The condition diagnosis according to this embodiment is divided into a diagnosis process using real-time images and a detailed diagnosis process using captured still images. Each of these will be described below.

[0028] (Real-time diagnostic processing) In real-time diagnostic processing, the worker activates the photographing unit 103 of the terminal device 100 and photographs the appearance of the rolling device or its components to be diagnosed as a real-time image. Information about damage occurring in the photographed rolling device is then superimposed on the real-time image and displayed. Furthermore, various notifications are given depending on the photographing status of the real-time image. The following describes this process using a UI screen and a flowchart.

[0029] 2 is a flowchart of real-time diagnostic processing according to this embodiment. This processing flow is realized by the processing unit 101 of the terminal device 100 reading and executing various programs and data stored in the storage unit 102. In addition, before this processing flow is started, a trained model is made available through the training process by the server device 200. In addition, when this processing flow is started, the imaging unit 103 of the terminal device 100 is in an activated state.

[0030] In S201, the terminal device 100 derives an imaging range according to the diagnosis target. For example, information about the diagnosis target may be acquired by accepting designation of the product type, individual components, etc. via a UI screen described below. Then, the terminal device 100 derives the range to be imaged in the condition diagnosis based on the information about the diagnosis target. The imaging range may be, for example, the target area to be imaged, such as the surface of the rolling elements constituting the rolling device or the rolling surfaces of the inner and outer rings. Alternatively, the imaging range may be the range of one circumference of the rolling surface, or the number of still images to be captured.

[0031] In S202, the terminal device 100 acquires real-time images via the image capturing unit 103. The real-time images are continuously acquired while the image capturing unit 103 is running. At this time, the terminal device 100 may display information about the object to be captured and information about navigation (such as a frame) on a UI screen displayed on the UI unit 104, based on the image capturing range derived in S201.

[0032] In S203, the terminal device 100 inputs the real-time image acquired in S202 into the trained model to detect the area of ​​the target part. The trained model used here may be selected from multiple models based on the information specified in S201 (such as the product type).

[0033] In S204, the terminal device 100 determines whether or not image capture adjustment is necessary based on the output from the trained model. Specifically, if the output of the trained model determines that the area to be diagnosed cannot be detected from the real-time image, the terminal device 100 may determine that image capture adjustment is necessary. If it is determined that image capture adjustment is necessary (YES in S204), the processing of the terminal device 100 proceeds to S205. On the other hand, if it is determined that image capture adjustment is not necessary (NO in S204), the processing of the terminal device 100 proceeds to S206.

[0034] In S205, the terminal device 100 displays instructions on the UI screen of the UI unit 104 to adjust the imaging conditions so that the diagnostic target is included in the real-time image. The imaging conditions may be, for example, the angle of view, direction, brightness, distance, etc., and the operator is prompted to adjust these. In this case, the instructions may be given by displaying text, icons, etc. Then, the processing of the terminal device 100 returns to S202 and continues.

[0035] In S206, the terminal device 100 performs a condition diagnosis of the rolling device using real-time images based on the output from the trained model. For convenience, information regarding the condition of the part output from the trained model is referred to as condition information. The condition information here may include, for example, the location and type of abnormality. The type of abnormality may be peeling, peeling, wear, or the like, or the diagnosis result may be that there is no abnormality.

[0036] In S207, the terminal device 100 displays the diagnosis result superimposed on the real-time image. Fig. 4 shows an example of the configuration of a UI screen 400 displayed in this step. The UI screen 400 will be described in detail later.

[0037] In S208, the terminal device 100 determines whether or not an instruction to record the diagnosis results has been received. The instruction to record may be received, for example, by pressing a button or the like provided on the terminal device 100. If the instruction to record has been received (YES in S208), the processing of the terminal device 100 proceeds to S209. On the other hand, if the instruction to record has not been received (NO in S208), the processing of the terminal device 100 proceeds to S210.

[0038] In S209, the terminal device 100 records the real-time image as a still image in association with the condition diagnosis result (condition information) obtained in S206. The condition information may include product information about the product to be diagnosed, information about the time the image was taken, etc. The process then proceeds to S210.

[0039] In S210, the terminal device 100 determines whether images covering the imaging range derived in S201 have been acquired. The images may be real-time images or still images captured in S209. For example, the imaging target may be the inner wheel of a rolling device. In this case, if a imaging range in which at least two images of the inner wheel are captured from different angles is derived in S201, the terminal device 100 determines whether two or more still images have been recorded in S209. Alternatively, if a imaging range in which real-time images covering one revolution of the inner wheel are acquired is derived in S201, the terminal device 100 determines whether real-time images covering one revolution of the inner wheel have been acquired in S202. The acquisition of a predetermined range using real-time images may be determined based on, for example, marks provided at predetermined positions on the inner wheel. Alternatively, the terminal device 100 may determine whether point cloud data covering one revolution of the inner wheel has been acquired using point cloud data obtained using a distance sensor or the like provided in the terminal device 100. If images covering the predetermined imaging range have been acquired (YES in S210), the processing of the terminal device 100 proceeds to S211. On the other hand, if images for the predetermined shooting range have not been acquired (NO at S210), the process of terminal device 100 proceeds to S212.

[0040] In S211, the terminal device 100 displays a message indicating that the diagnostic object has reached a predetermined imaging range on the UI screen of the UI unit 104. After that, the process of the terminal device 100 proceeds to S212.

[0041] In S212, the terminal device 100 determines whether or not image capture has ended. For example, if an instruction to end image capture is received from the operator via the UI unit 104, the terminal device 100 may determine that image capture has ended. If image capture has ended (YES in S212), the process flow ends. On the other hand, if image capture has not ended (NO in S212), the process of the terminal device 100 returns to S202 and the process is repeated.

[0042] (Detailed diagnosis process) In the detailed diagnosis process, detailed diagnosis results are presented using still images captured during real-time diagnosis process or the like. In this embodiment, an example is shown in which still images captured during the real-time diagnosis process shown in Fig. 2 are used. However, the present invention is not limited to this, and a still image of a component acquired in advance as the diagnosis target may also be used.

[0043] 3 is a flowchart of the detailed diagnosis process according to this embodiment. This process flow is realized by the processing unit 101 of the terminal device 100 reading and executing various programs and data stored in the storage unit 102.

[0044] In S301, the terminal device 100 accepts the selection of an image to be diagnosed via a UI screen displayed on the UI unit 104. Furthermore, the terminal device 100 accepts input of information related to the selected image. Fig. 5 shows an example of the configuration of a UI screen 500 for accepting the image selection. Details of the UI screen 500 will be described later.

[0045] In S302, the terminal device 100 identifies a damage mode based on the status information associated with the image selected in S301. The status information here corresponds to the status information output by the trained model in S206 of FIG. 2. If there is no status information associated with the selected still image, new status information may be acquired using the trained model with the selected still image as input. Examples of the identified damage mode include "peeling," "peeling," and "wear."

[0046] In S303, the terminal device 100 determines whether additional measurements are necessary based on the damage mode identified in S302. For example, depending on the damage mode, performing additional measurements on the diagnosed object may enable a more detailed condition diagnosis. On the other hand, depending on the damage mode, the appropriate action by the operator may be determined without performing additional measurements. In this embodiment, a table is used that predefines whether additional measurements are necessary depending on the identified damage mode and damaged location. FIG. 8 shows an example of a correspondence table 800 used in this process. In this example, damaged parts are either "repaired" or "replaced." Undamaged parts can be "reused." Furthermore, if the damage mode allows for repair, additional measurements are performed. Note that this is just one example, and the corresponding actions may be defined depending on the characteristics of the diagnosed object. For example, if replacement is more cost-effective than repair for a part, the correspondence table 800 may be defined so that replacement is prioritized. If additional measurement is necessary (YES in S303), the process of the terminal device 100 proceeds to S304. On the other hand, if additional measurement is not necessary (NO in S303), the process of the terminal device 100 proceeds to S308.

[0047] In S304, the terminal device 100 acquires information about the additional measurement items and measurement methods corresponding to the damage mode. This information is predefined in association with the damage mode, and in this embodiment, a correspondence table is used. FIG. 9 shows an example of the correspondence table 900 used in this step. Furthermore, the information about the measurement method includes information for clearly indicating the specific measurement method. Specifically, examples include a video illustrating the measurement and an image showing the measurement position.

[0048] FIG. 7 shows an example of a specific video 700 during measurement. For example, assume that the peeling mode is "peeling" and the damaged location is "inner ring, raceway surface." In this case, in the correspondence table 900, the additional measurement item is "circumferential length" and the measurement method is "caliper." A threshold value is also specified for this. The threshold value is used in step S307, which will be described later. Then, a video in which an example of additional measurement using a caliper is recorded is specified as lecture information for clearly indicating the specific measurement method. The video 700 shown in FIG. 7 is an example in which a target location (item) of an object 701 is measured using a caliper. Lecture information such as the video 700 is registered in advance depending on the additional measurement, diagnosis target, etc.

[0049] In S305, the terminal device 100 presents the various pieces of information acquired in S304 via a UI screen of the UI unit 104. Fig. 6 shows an example of the configuration of a UI screen 600 displayed in this step. Details of the UI screen 600 will be described later.

[0050] In S306, terminal device 100 accepts input of the results of the additional measurement via UI screen 600. That is, the operator inputs the actual measurement values ​​obtained by the additional measurement via UI screen 600.

[0051] In S307, the terminal device 100 performs a condition diagnosis using the input results of the additional measurement. In this embodiment, the action to be taken is determined using the threshold value corresponding to the damage mode acquired in S304 and the actual measurement value of the additional measurement received in S306. For example, if the actual measurement value is smaller than the threshold, repair is performed, and if the actual measurement value is equal to or greater than the threshold, replacement is performed. Note that the treatment of the threshold value acquired in S304 may differ depending on the type of component or the damage mode.

[0052] In S308, the terminal device 100 performs a condition diagnosis using the acquired measurement results. Here, no additional measurements are performed, and the terminal device 100 may identify, for example, a damage mode and a response (in this example, "replacement" of the diagnostic target) as the condition diagnosis results.

[0053] In S309, the terminal device 100 outputs the diagnosis result of S307 or S308 via the UI screen of the UI unit 104. Fig. 6 shows an example of a UI screen 600 displayed in this step. Details of the UI screen 600 will be described later. Then, this processing flow ends.

[0054] (UI Screen) An example of the configuration of a UI screen displayed on the terminal device 100 according to this embodiment will be described using Figures 4 to 6. Figure 4 shows an example of the configuration of a UI screen displayed during the real-time image diagnosis process of Figure 2. Figures 5 and 6 show an example of the configuration of a UI screen displayed during the detailed diagnosis process of Figure 3.

[0055] A UI screen 400 shown in Fig. 4 displays a real-time image of an inner ring 401 of a rolling device, which is the target of diagnosis. Furthermore, by inputting the real-time image into the trained model described above, the damaged area and its type (damage mode) are identified. In this example, damage in two locations on the rolling surface of the inner ring 401 and its damage mode, "peeling," have been identified. Furthermore, icons 402 and 403 indicating the identified information are superimposed on the real-time image.

[0056] By displaying the results of the condition diagnosis in real time, as in the UI screen 400, the operator can easily recognize the outline of the condition of the diagnosis target. Although not shown in Fig. 4, instructions and notifications in steps S205 and S211 of Fig. 2 may be superimposed and displayed on the UI screen 400. Furthermore, the instruction to take a photograph in step S208 of Fig. 2 may be received using a conventional photographing function provided in the terminal device 100. Furthermore, as one type of conventional photographing function, a display related to dimensions and zooming in / out may be superimposed on the real-time image.

[0057] Next, the UI screen used in the detailed diagnosis process will be described. The UI screen 500 shown in FIG. 5 includes a menu button 501. When the menu button 501 is selected, a selection screen (not shown) for selecting the rolling device to be diagnosed is displayed. Information about the rolling device selected on the selection screen (not shown) is displayed in a bar 502. This selection screen may be configured to display a list of diagnosable products so that the operator can select one. Furthermore, the information entered on this selection screen may be used in the real-time diagnosis process, as obtained, for example, in step S201 of FIG. 2. In this example, the bar 502 displays "Company AAA" indicating the manufacturer of the rolling device and "XXXXX" indicating product information.

[0058] When the camera icon 503 is selected, the screen transitions to the UI screen 400 shown in Fig. 4. That is, when the camera icon 503 is selected, a real-time diagnostic process may be performed. When the folder icon 504 is selected, a selection screen (not shown) for selecting a still image that has already been captured is displayed. This selection screen may be configured to allow selection of a still image captured during the real-time diagnostic process.

[0059] Still image 505 indicates a still image selected as an image of the diagnosis target. In this example, a still image of the inner ring of a rolling device is selected and displayed. Settings 506, 507, and 508 are parameters for the operator to set detailed information of the diagnosis target. In this example, the model number of the rolling device is input into setting 506. The part that constitutes the rolling device, i.e., the type of part shown in still image 505, is input into setting 507. The part of the part that is the diagnosis target is input into setting 508. In this example, the model number "XXXXX", the part "inner ring", and the diagnosis target "rolling surface" are input. The settings may be configured to be selectable in a list format or may be configured to be manually input. The number and types of setting items may vary depending on the product, etc.

[0060] When the diagnosis button 509 is pressed, detailed diagnosis processing is executed. That is, processing from S302 onward in Fig. 3 is started. The status icon 510 indicates the current status of the detailed diagnosis processing. In this case, "Setting," which indicates a state in which setting input is accepted, may be indicated by lighting up or the like.

[0061] In a UI screen 600 shown in Fig. 6, a menu button 601 is similar to the menu button 501 on the UI screen in Fig. 5. A diagnosis result image 602 displays status information associated with the still image 505 in Fig. 5. In this example, two damaged areas and their damage mode "peeling" are identified and displayed as icons 603 and 604.

[0062] In this example, it is assumed that an additional measurement is required. Field 605 is an item for inputting the actual measurement value of the additional measurement. When the method button 606 is pressed, the method of the additional measurement is displayed. Specifically, a moving image 700 such as that shown in FIG. 7 is displayed. The content displayed when the method button 606 is pressed varies depending on the content of the additional measurement. When the actual measurement value is entered in field 605 and the input button 607 is pressed, a condition diagnosis is performed using the actual measurement value of the additional measurement. This corresponds to step S307 in FIG. 3 . The result is then displayed in result 608. Note that if it is determined that an additional measurement is not required, the field 605, method button 606, and input button 607 may be hidden. If multiple additional measurements are required, multiple fields 605 and method buttons 606 may be displayed corresponding to each additional measurement. If the condition diagnosis determines that no abnormality is present, the result 608 may indicate this, or may display "reuse."

[0063] The response information 609 indicates information on the case where a response related to the result of the status diagnosis is taken. For example, if the result 608 indicates "repair," information on the effect obtained by actually performing the repair may be displayed. Specifically, the information indicated in the response information 609 may include environmental information (such as CO2 reduction amount) and costs (the difference between repair and replacement).

[0064] When the transition button 610 is pressed, the user is redirected to a website where the part to be repaired can be purchased or to the website of a company that undertakes the replacement work. The destination of the transition here may be switched based on the product information entered via the UI screen. The status icon 611 indicates the current status of the detailed diagnosis process, and in this case, "Results," which indicates that the diagnosis results have been output, may be indicated by lighting up.

[0065] 4 to 6 are merely examples, and the present invention is not limited to these. For example, the displayed items may be changed depending on the product to be diagnosed.

[0066] In the configuration of FIG. 3 , if it is determined in step S303 that additional measurements are necessary, the actual measurement values ​​from the additional measurements are input and then a condition diagnosis is performed. However, this configuration is not limited to this. For example, even if it is determined that additional measurements are necessary, a condition diagnosis may be performed based on information acquired at that time, and a simplified diagnosis result may be output. For example, if additional measurements of the damage length are required using a caliper, and the dimensions of the damage can be estimated from acquired still images or video images, a simplified diagnosis result may be output using the estimated dimensions of the damage. This allows the operator to understand the results of the condition diagnosis at that time before performing the additional measurements. This is useful, for example, when additional measurements are time-consuming or laborious.

[0067] 8 and 9 may be held on the terminal device 100 side or on the server device 200 side. When the terminal device 100 makes a diagnosis, the terminal device 100 may refer to the information defined in the correspondence tables 800 and 900 in the database by making an inquiry to the server device 200. In addition, the terminal device 100 may also obtain information related to the results of the condition diagnosis, such as information such as the moving image 700, by making an inquiry to the server device 200.

[0068] As described above, this embodiment makes it possible to realize a condition diagnosis method for mechanical devices that is easy to implement and has higher usability. In particular, even non-experts can easily grasp the condition of the mechanical device and smoothly take measures according to the condition.

[0069] <Other Embodiments> Furthermore, in the present invention, a program or application for realizing the functions of one or more of the above-described embodiments can be supplied to a system or device using a network or a storage medium, etc., and one or more processors in a computer of the system or device can read and execute the program.

[0070] Alternatively, it may be realized by a circuit that realizes one or more functions (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

[0071] Furthermore, when the terms "first" and "second" are used in the description of this specification, they are used merely for convenience to distinguish from other elements and are not intended to be interpreted in a limiting manner as specific elements. Therefore, these terms should be interpreted appropriately depending on the combination and number of components.

[0072] As such, the present invention is not limited to the above-described embodiments, and the present invention also contemplates the mutual combination of the various components of the embodiments, as well as modifications and applications by those skilled in the art based on the description in the specification and well-known techniques, and these modifications and applications are included in the scope of protection sought.

[0073] As described above, the present specification discloses the following: (1) A condition diagnosis system (e.g., 100) including: acquisition means (e.g., 101, 103) for acquiring an image including a diagnosis target; identification means (e.g., 101) for identifying a damaged portion and a type of damage in the image acquired by the acquisition means using a trained model that receives the image as input and outputs a damaged portion and a type of damage of the diagnosis target contained in the image; and output means (e.g., 101, 102, 104, 400) for outputting the image acquired by the acquisition means in association with the damaged portion and the type of damage identified by the identification means. This configuration enables a condition diagnosis method for mechanical equipment that is easy to implement and has higher usability. In particular, even non-experts can easily grasp the condition of the mechanical equipment and smoothly take appropriate action according to the condition.

[0074] (2) The condition diagnosis system according to (1), further comprising a diagnosis unit that identifies a response to the diagnosis target using the damaged portion and type of damage in the image acquired by the acquisition unit and outputs the response to the diagnosis target on a user interface screen (e.g., 600). This configuration makes it possible to realize a condition diagnosis method for mechanical devices that is easy to implement and has higher usability. In particular, even non-experts can easily grasp the condition of the mechanical device and smoothly take appropriate action according to the condition.

[0075] (3) The condition diagnosis system according to (2), further comprising: a determination means (e.g., 101) for determining whether additional measurements are necessary for the diagnostic object based on the damaged portion and type of damage identified by the identification means; a determination means (e.g., 101) for determining information on additional measurements for the diagnostic object according to the damaged portion and type of damage identified by the identification means when the determination means determines that additional measurements are necessary; and a display control means (e.g., 101) for displaying the information on the additional measurements determined by the determination means and a field for inputting actual measurement values ​​obtained from the additional measurements on the user interface screen (e.g., 600). The diagnosis means further uses the actual measurement values ​​to identify a response to the diagnostic object and output the response to the user interface screen (e.g., 608). This configuration allows an operator to easily understand the items of additional measurements to be performed on the diagnostic object. It also improves operability when inputting actual measurement values.

[0076] (4) The condition diagnosis system according to (3), wherein the information on the additional measurements includes lecture information (e.g., 700) on the additional measurements for the diagnosis target. With this configuration, an operator can easily understand the methods of additional measurements required for the machine.

[0077] (5) The condition diagnosis system according to any one of (2) to (4), wherein the response includes at least one of repair and replacement of the machine to be diagnosed. With this configuration, it is possible to easily determine whether to repair or replace the damaged machine.

[0078] (6) The condition diagnosis system according to (5), wherein the response further includes determining that the machine is reusable. With this configuration, it is possible to present whether the machine is reusable depending on its condition, and the worker can easily understand that the machine is reusable.

[0079] (7) The condition diagnosis system according to any one of (2) to (6), wherein the diagnosis means further outputs information on a case where a countermeasure is taken for the diagnosis target to the user interface screen (e.g., 609). With this configuration, it becomes possible to easily grasp the effect of taking the countermeasure presented for the damaged mechanical device.

[0080] (8) The condition diagnosis system according to any one of (1) to (7), wherein the diagnosis target is a rolling device. With this configuration, it is possible to realize a condition diagnosis method for a rolling device that can be easily introduced and has higher usability.

[0081] (9) A condition diagnosis method including: an acquisition step (e.g., S202, S301) of acquiring an image including a diagnosis target; an identification step (e.g., S206, S302) of identifying a damaged portion and a type of damage in the image acquired in the acquisition step using a trained model that receives the image as input and outputs a damaged portion and a type of damage of the diagnosis target contained in the image; and an output step (e.g., S207, S209, S302) of correlating the image acquired in the acquisition step with the damaged portion and the type of damage identified in the identification step and outputting the image. This configuration makes it possible to realize a condition diagnosis method for mechanical devices that is easy to implement and has higher usability. In particular, even non-experts can easily grasp the condition of the mechanical device and smoothly take appropriate action according to the condition.

[0082] (10) A program for causing a computer (e.g., 100) to execute: an acquisition unit (e.g., 101) for acquiring an image including a diagnosis target; an identification unit (e.g., 101) for identifying a damaged portion and a type of damage in the image acquired by the acquisition unit using a trained model that receives an image as input and outputs a damaged portion and a type of damage of the diagnosis target contained in the image; and an output unit (e.g., 101) for outputting the image acquired by the acquisition unit in association with the damaged portion and the type of damage identified by the identification unit. This configuration makes it possible to realize a condition diagnosis method for mechanical equipment that is easy to implement and has higher usability. In particular, even non-experts can easily grasp the condition of the mechanical equipment and smoothly take measures according to the condition.

[0083] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0084] This application is based on a Japanese patent application (Patent Application No. 2023-210073) filed on December 13, 2023, the contents of which are incorporated herein by reference.

[0085] REFERENCE SIGNS LIST 100: Terminal device 101: Processing unit 102: Storage unit 103: Photography unit 104: UI unit 105: Communication unit 200: Server device 201: Processing unit 202: Storage unit 203: Communication unit 204: Input / output unit 300: Network

Claims

1. A condition diagnosis system having: an acquisition means for acquiring an image including a diagnostic object; an identification means for identifying a damaged portion and a type of damage in the image acquired by the acquisition means using a trained model that takes an image as input and outputs the damaged portion and the type of damage of the diagnostic object contained in the image; and an output means for outputting the image acquired by the acquisition means in association with the damaged portion and the type of damage identified by the identification means.

2. A condition diagnosis system according to claim 1, further comprising a diagnosis means for identifying a response to the diagnosis object using the damaged portion and type of damage in the image acquired by the acquisition means, and outputting the response to the diagnosis object on a user interface screen.

3. A condition diagnosis system as described in claim 2, comprising: a judgment means for judging whether or not additional measurement is necessary for the diagnostic object based on the damaged part and type of damage identified by said identification means; a decision means for deciding information on additional measurement for the diagnostic object according to the damaged part and type of damage identified by said identification means when said judgment means decides that additional measurement is necessary; and a display control means for displaying on said user interface screen a field for inputting the information on additional measurement decided by said decision means and an actual measurement value obtained by the additional measurement, wherein said diagnosis means further uses the actual measurement value to identify a response to the diagnostic object and output it to said user interface screen.

4. A condition diagnosis system as described in claim 3, wherein the additional measurement information includes lecture information of an additional measurement for the diagnostic object.

5. A condition diagnosis system as described in claim 2, wherein the response includes at least one of repairing and replacing the object to be diagnosed.

6. The condition diagnosis system according to claim 5, wherein the response further includes determining reuse.

7. The condition diagnosis system according to claim 2, wherein said diagnosis means further outputs information on a case where a measure has been taken against said diagnosis target to said user interface screen.

8. The condition diagnosis system according to claim 1, wherein the object to be diagnosed is a rolling device.

9. A condition diagnosis method comprising: an acquisition step of acquiring an image including a diagnostic object; an identification step of identifying a damaged portion and a type of damage in the image acquired in the acquisition step using a trained model that receives an image as input and outputs a damaged portion and a type of damage of the diagnostic object contained in the image; and an output step of outputting the image acquired in the acquisition step in association with the damaged portion and the type of damage identified in the identification step.

10. A program for causing a computer to execute the following: an acquisition means for acquiring an image including a diagnostic object; an identification means for identifying a damaged portion and a type of damage in the image acquired by the acquisition means using a trained model that takes an image as input and outputs the damaged portion and type of damage of the diagnostic object contained in the image; and an output means for outputting the image acquired by the acquisition means in association with the damaged portion and type of damage identified by the identification means.

Images