Condition diagnosis system, condition diagnosis method, and program

Abstract

This state diagnosis system includes: an acquisition means for acquiring an image including a diagnosis target; a specification means for specifying a damaged portion and type of damage of the diagnosis target in the image acquired by the acquisition means by using a trained model that receives input of an image to output a damaged portion and type of damage of a diagnosis target included in the image; and an output means for outputting the image acquired by the acquisition means and the damaged portion and type of damage specified by the specification means in association with each other.

Description

Technical Field

[0001] The present invention relates to a state diagnosis system, a state diagnosis method, and a program.

Background Art

[0002] Conventionally, various components are used in mechanical devices, and by appropriately managing the states of these components, stable operation of the mechanical devices is achieved. For the mechanical devices, the state diagnosis of the components is performed by visual inspection by an operator or the like, and after confirming the state of the components, replacement or repair is carried out.

[0003] Such work depends on the skill level and experience of the operator, so it has not been easy to perform the state diagnosis of mechanical devices. In contrast, for example, in Patent Document 1, a method of acquiring a photographed image of an external component of a vehicle and determining damage to the external component using a component learning model and a state learning model is disclosed. Also, in Patent Document 2, a method of acquiring an image of the tread surface of a tire and diagnosing the degree of uneven wear of the tire using a learned AI is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, consider a bearing system as a mechanical device where parts can be replaced. Damage to each component of a bearing system can manifest in various ways. To perform a more accurate condition diagnosis, additional condition information may be needed in addition to the appearance of the parts. For example, a skilled worker can determine what additional measurements or information acquisition is necessary, based on the information currently obtained. Furthermore, a skilled worker can determine whether replacement or repair is the appropriate course of action in the current situation. However, inexperienced workers may not be able to adequately determine what action should be taken during a condition diagnosis. In such cases, methods such as those described in Patent Document 1 cannot be used as is.

[0006] In view of the above issues, the present invention aims to realize a condition diagnosis method for mechanical devices that can be easily implemented and has higher usability. [Means for solving the problem]

[0007] To solve the above problems, the present invention has the following configuration. That is, the condition diagnosis system is A means for acquiring images that include the subject of diagnosis, An identification means that identifies the damaged area and type of damage in an image acquired by the acquisition means, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image, An output means that outputs an image acquired by the acquisition means and the damaged portion and type of damage identified by the identification means, It holds.

[0008] Another embodiment of the present invention has the following configuration. That is, the condition diagnosis method is The acquisition process involves obtaining images that include the subject of diagnosis, A selection step involves using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image to be identified in the acquisition step, An output step that outputs the image acquired in the acquisition step, along with the damaged area and type of damage identified in the identification step, It holds.

[0009] Another embodiment of the present invention has the following configuration. That is, the program is On the computer, A means for acquiring images that include the subject of diagnosis, An identification means that identifies the damaged area and type of damage in an image acquired by the acquisition means, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image, An output means that outputs an image acquired by the acquisition means and the damaged portion and type of damage identified by the identification means, Make it run. [Effects of the Invention]

[0010] This invention makes it possible to realize a condition diagnosis method for mechanical devices that can be easily implemented and has higher usability. [Brief explanation of the drawing]

[0011] [Figure 1] A schematic diagram showing an example of a system configuration related to one embodiment of the present invention. [Figure 2] Flowchart of real-time diagnostic processing related to Embodiment 1. [Figure 3] Flowchart of the detailed diagnostic process for one embodiment of the present invention. [Figure 4] A diagram showing an example of the UI configuration for image acquisition in one embodiment of the present invention. [Figure 5] A diagram showing an example configuration of a UI for condition diagnosis according to one embodiment of the present invention. [Figure 6] A diagram showing an example configuration of a UI for condition diagnosis according to one embodiment of the present invention. [Figure 7] A diagram showing an example of content for additional measurements related to one embodiment of the present invention. [Figure 8] A diagram showing an example of a correspondence table for state diagnosis according to an embodiment of the present invention. [Figure 9] A diagram showing an example of a correspondence table for state diagnosis according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings and the like. The embodiments described below are one embodiment for explaining the present invention, and are not intended to be construed as limiting the present invention. Also, not all the configurations described in each embodiment are necessarily essential configurations for solving the problems of the present invention. In each drawing, the same reference numerals are assigned to the same components to indicate the correspondence relationship.

[0013] In the following description, "learning" or "machine learning" refers to generating a "trained model" by repeatedly performing learning processing using learning data and an arbitrary learning algorithm. The trained model is updated in a timely manner as learning progresses using a plurality of learning data, and its output changes even for the same input. Therefore, the trained model does not limit the state at any point in time. Here, the model used in learning is described as a "learning model", and a learning model that has undergone a certain degree of learning is described as a "trained model".

[0014] Specific examples of "learning data" will be described later, but its configuration may be adjusted and changed according to the learning algorithm to be used, its application, and the type of device to be diagnosed. For example, in addition to the image data described later, data such as vibration data, sound data, ultrasonic data, and numerical data may be used as learning data. Also, the configuration of the learning data and the preprocessing for the learning data can vary according to the type of learning algorithm and the content of the input and output of the learning model.

[0015] Furthermore, training data may include training data used for training itself, validation data used to validate the trained model, and test data used to test the trained model. In the following explanation, "training data" will be used to refer comprehensively to data related to training. Note that this is not intended to clearly classify the training data into training data, validation data, and test data; for example, depending on the training, validation, and testing methods, all of the training data may also be training data.

[0016] <First Embodiment> The first embodiment of the present invention will be described below. In this embodiment, a bearing device will be used as an example of the device to be diagnosed. However, the present invention is applicable to any device that performs diagnostics such as replacement and repair of parts, as will be described later.

[0017] [System Configuration] The following describes one embodiment of a system to which the method according to this embodiment can be applied. Figure 1 is a schematic diagram showing an example of the system configuration according to this embodiment. As shown in Figure 1, the system according to this embodiment is configured to include a terminal device 100 and a server device 200, which are connected to each other so as to be able to communicate via a network 300. In this embodiment, one terminal device 100 and one server device 200 are used as examples, but the system is not limited to these. Multiple units of each device may be provided, and they may cooperate by sharing various processing tasks according to their functions.

[0018] The terminal device 100 is a device that can be used around the bearing device (not shown) to be diagnosed, and may include, for example, a smartphone, tablet terminal, POS terminal, or dedicated device. The terminal device 100 consists of a processing unit 101, a storage unit 102, an imaging unit 103, a UI (User Interface) unit 104, and a communication unit 105. The processing unit 101 may consist of a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a dedicated circuit.

[0019] The storage unit 102 is composed of volatile and non-volatile storage media such as HDD (Hard Disk Drive), ROM (Read Only Memory), RAM (Random Access Memory), and flash memory, and can input and output various types of information according to instructions from the processing unit 101. The imaging unit 103 is, for example, a camera, and can acquire real-time images and still images of the bearing device that is the target of diagnosis. The UI unit 104 displays a UI screen, which will be described later, and accepts operations from the user. The UI unit 104 may be composed of, for example, a touch panel display. The communication unit 105 is a communication interface that controls communication with an external device (for example, a server device 200). The communication standard by the communication unit 105 via the network 300 is not particularly limited, and multiple communication standards may be combined.

[0020] The server device 200 is an information processing device that works in conjunction with the terminal device 100 to provide the functions described later. The server device 200 may be built as an on-premise type in the environment in which the terminal device 100 is used, or it may be built as a cloud type on the internet. The network 300 may be configured by combining, for example, a LAN (Local Area Network), a WAN (Wide Area Network), and the internet.

[0021] The server device 200 comprises a processing unit 201, a storage unit 202, a communication unit 203, and an input / output unit 204. The processing unit 201 may consist of a CPU, GPU, GPGPU, MPU, DSP, FPGA, or dedicated circuitry. For example, the processing unit 201 of the server device 200 may be configured to be suitable for training processing to generate a pre-trained model, as described later.

[0022] The storage unit 202 is composed of volatile and non-volatile storage media such as HDDs, ROMs, and RAMs, and can input and output various types of information according to instructions from the processing unit 201. The communication unit 203 is a communication interface that controls communication with external devices (e.g., terminal devices 100). The communication standard by the communication unit 203 via the network 300 is not particularly limited, and multiple communication standards may be combined. The input / output unit 204 is an interface for inputting and outputting various types of data and instructions.

[0023] In the system according to this embodiment, it is assumed that an operator (for example, a manager of the rolling mechanism) will use the terminal device 100 to diagnose the condition of the rolling mechanism. The functions described later may be provided through cooperation between the terminal device 100 and the server device 200. Note that the division of processing shown below is just an example, and the main processing unit may be changed depending on the applicable environment, etc. Furthermore, the functions described later may be implemented as an application installed on the terminal device 100, or as a web application provided by the server device 200, or a combination of these.

[0024] Generally, the training process for generating pre-trained models is computationally intensive. Therefore, the pre-trained models used in this embodiment are described as being generated by a training process performed at predetermined timings on the server device 200. At this time, the server device 200 collects training data as appropriate and repeats the training process. The server device 200 maintains and manages multiple pre-trained models according to the degree of training and the type of rolling device to be diagnosed. Then, the server device 200 provides the pre-trained models to the terminal device 100 at predetermined timings, so that the terminal device 100 can use the pre-trained models during the diagnostic process.

[0025] The trained model according to this embodiment is a trained model that takes an image containing parts constituting a rolling device as input, identifies the damaged parts and types of damage from the image, and outputs them. The training algorithm here is not particularly limited, but for example, known deep learning methods such as R-CNN (Region Convolutional Neural Network), YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector) may be used.

[0026] Furthermore, the training data used in the learning process consists of image data to which annotation data indicating the damaged areas and their types are associated. The training data may be generated manually or using a predetermined tool. The generation of the training data itself can be done using publicly known methods, and a detailed explanation is omitted here.

[0027] [Processing flow] The following describes the state diagnosis process according to this embodiment. The state diagnosis according to this embodiment is divided into a diagnosis process using real-time images and a detailed diagnosis process using captured still images. Each of these will be described below.

[0028] (Real-time diagnostic processing) In the real-time diagnostic process, the operator activates the imaging unit 103 of the terminal device 100 to capture a real-time image of the rolling mechanism or its components that are to be diagnosed. Then, information regarding the damage occurring in the captured rolling mechanism is superimposed on the real-time image and displayed. Furthermore, various notifications are given according to the status of the real-time image capture. The following will be explained using the UI screen and flowchart.

[0029] Figure 2 is a flowchart of the real-time diagnostic process according to this embodiment. This process flow is realized by the processing unit 101 of the terminal device 100 reading and executing various programs and data stored in the storage unit 102. In addition, before this process starts, the trained model is available through the training process performed by the server device 200. Also, when this process flow starts, the imaging unit 103 of the terminal device 100 is activated.

[0030] In S201, the terminal device 100 derives the imaging range according to the diagnostic target. For example, information regarding the diagnostic target may be obtained by accepting specifications such as product type and individual parts via a UI screen described later. Then, the terminal device 100 derives the range to be photographed in the condition diagnosis based on the information regarding the diagnostic target. The imaging range may be, for example, the surface of the rolling elements constituting the rolling gear, or the rolling surfaces of the inner and outer rings, which are the target areas to be photographed. Alternatively, the imaging range may be the area of ​​one full rotation of the rolling surface, or the number of still images to be taken.

[0031] In S202, the terminal device 100 acquires real-time images via the imaging unit 103. Real-time images are continuously acquired while the imaging unit 103 is running. At this time, the terminal device 100 may display information about the object to be photographed and information related to navigation (such as a frame) on the UI screen displayed on the UI unit 104, based on the shooting range derived in S201.

[0032] In S203, the terminal device 100 inputs the real-time image acquired in S202 into a trained model to detect the region of the target component. The trained model used here may be selected from several options based on the information specified in S201 (such as product type).

[0033] In S204, the terminal device 100 determines whether or not adjustments to the image capture are necessary based on the output from the trained model. Specifically, if the output from the trained model determines that the area to be diagnosed cannot be detected from the real-time image, it may be determined that adjustments to the image capture are necessary. If it is determined that adjustments to the image capture are necessary (YES in S204), the terminal device 100 proceeds to S205. On the other hand, if it is determined that adjustments to the image capture are not necessary (NO in S204), the terminal device 100 proceeds to S206.

[0034] In S205, the terminal device 100 displays instructions on the UI screen of the UI unit 104 to adjust the shooting conditions so that the object to be diagnosed is included in the real-time image. Shooting conditions may include, for example, field of view, orientation, brightness, and distance, and the instructions prompt the operator to adjust these. These instructions may be given by displaying text or icons. Afterward, the terminal device 100 returns to S202 and continues processing.

[0035] In S206, terminal device 100 performs a real-time image-based diagnosis of the rolling mechanism's condition based on the output from the trained model. For convenience, the information regarding the condition of the parts, which is the output from the trained model, is referred to as condition information. This condition information may include, for example, the location and type of any abnormality. The type of abnormality may be delamination, peeling, wear, etc., or the diagnosis may simply be that there is no abnormality.

[0036] In step S207, the terminal device 100 displays the diagnostic results superimposed on the real-time image. Figure 4 shows an example of the configuration of the UI screen 400 displayed in this process. Details of the UI screen 400 will be described later.

[0037] In S208, the terminal device 100 determines whether or not it has received an instruction to record the diagnostic results. The instruction to record may be received, for example, by pressing a button on the terminal device 100. If the instruction to record has been received (YES in S208), the terminal device 100 proceeds to S209. On the other hand, if the instruction to record has not been received (NO in S208), the terminal device 100 proceeds to S210.

[0038] In S209, terminal device 100 records the real-time image as a still image, associating it with the status diagnosis result (status information) from S206. The status information may include information about the product being diagnosed and the time the image was taken. After that, terminal device 100 proceeds to S210.

[0039] In S210, the terminal device 100 determines whether or not it has acquired images for the shooting range derived in S201. The images here may be real-time images or still images taken in S209. For example, the object to be photographed may be the inner ring of a rolling device. In this case, if a shooting range was derived in S201 that allows for two images to be taken from at least two different angles relative to the inner ring, then in S209, it is determined whether or not two or more still images have been recorded. Alternatively, if a shooting range was derived in S201 that allows for the acquisition of a real-time image covering one full rotation of the inner ring, then in S202, it is determined whether or not a real-time image covering one full rotation of the inner ring has been acquired. The acquisition of a predetermined range using real-time images may be determined based on, for example, marks placed at predetermined positions on the inner ring. Alternatively, it may be determined whether or not point cloud data covering one full rotation of the inner ring has been acquired using point cloud data obtained using a distance sensor or the like provided by the terminal device 100. If images for the predetermined shooting range have been acquired (YES in S210), the processing of the terminal device 100 proceeds to S211. On the other hand, if images within the predetermined shooting range have not been acquired (NO in S210), the terminal device 100 proceeds to S212.

[0040] In S211, the terminal device 100 displays on the UI screen of the UI unit 104 that the target for diagnosis has reached the predetermined imaging range. After that, the processing of the terminal device 100 proceeds to S212.

[0041] In S212, the terminal device 100 determines whether or not the shooting is complete. For example, if it receives an instruction to end the shooting from the operator via the UI unit 104, it may determine that the shooting is complete. If the shooting is complete (YES in S212), this processing flow is terminated. On the other hand, if the shooting is not complete (NO in S212), the processing of the terminal device 100 returns to S202 and the process is repeated.

[0042] (Detailed diagnostic process) In the detailed diagnostic process, still images captured during the real-time diagnostic process are used to present detailed diagnostic results. In this embodiment, an example is shown in Figure 2, which uses still images captured during the real-time diagnostic process. However, this is not the only option, and still images of parts acquired in advance as the target of diagnosis may also be used.

[0043] Figure 3 is a flowchart of the detailed diagnostic process according to this embodiment. This process flow is realized when the processing unit 101 of the terminal device 100 reads and executes various programs and data stored in the storage unit 102.

[0044] In S301, the terminal device 100 accepts the selection of an image to be diagnosed via the UI screen displayed on the UI unit 104. Furthermore, the terminal device 100 accepts input of information regarding the selected image. Figure 5 shows an example configuration of the UI screen 500 for accepting image selection. Details of the UI screen 500 will be described later.

[0045] In S302, the terminal device 100 identifies the damage mode based on the state information associated with the image selected in S301. The state information here corresponds to the state information output by the trained model in S206 of Figure 2. If there is no state information associated with the selected still image, the trained model may be used to acquire new state information using the selected still image as input. Examples of damage modes that may be identified include "delamination," "peeling," and "wear."

[0046] In S303, the terminal device 100 determines whether additional measurements are necessary based on the damage mode identified in S302. For example, depending on the damage mode, it is possible to perform a more detailed condition diagnosis by performing additional measurements on the item to be diagnosed. On the other hand, depending on the damage mode, it may be possible to determine what the worker should do without performing additional measurements. In this embodiment, a table is used which pre-defines whether additional measurements are necessary according to the identified damage mode and the location of the damage. Figure 8 shows an example of the correspondence table 800 used in this process. In this example, for damaged parts, either "repair" or "replace" is performed. Parts that are not damaged can be "reused". If the damage mode allows for repair, additional measurements are performed. Note that this correspondence is just one example and may be defined according to the characteristics of the item to be diagnosed. For example, if a part can be replaced at a lower cost than repaired, the correspondence table 800 may be defined so that replacement is preferentially selected. If additional measurements are required (YES in S303), the terminal device 100 proceeds to S304. On the other hand, if additional measurements are not required (NO in S303), the terminal device 100 proceeds to S308.

[0047] In S304, the terminal device 100 acquires information regarding additional measurement items and measurement methods corresponding to the damage mode. This information is predefined and associated with the damage mode, and in this embodiment, a correspondence table is used. Figure 9 shows an example of the correspondence table 900 used in this process. Furthermore, the information regarding the measurement method includes information to clearly indicate how to perform the measurement. Specifically, this includes videos illustrating the measurement and images showing the measurement location.

[0048] Figure 7 shows a specific example of a video 700 used during measurement. For example, suppose the delamination mode is "peeling" and the damaged area is "inner ring, raceway surface". In this case, in the correspondence table 900, the additional measurement item is "circumferential length" and the measurement method is "caliper". A threshold value is also specified for this. The threshold value is used in process S307, which will be described later. Then, as lecture information to clarify the specific measurement method, a video recording an example of additional measurement using a caliper is identified. The video 700 shown in Figure 7 is an example of measuring the target area (item) of the object 701 using a caliper. Lecture information such as video 700 is registered in advance according to the additional measurement or diagnostic target.

[0049] In S305, the terminal device 100 presents the various information acquired in S304 via the UI screen of the UI unit 104. Figure 6 shows an example of the configuration of the UI screen 600 displayed in this process. Details of the UI screen 600 will be described later.

[0050] In S306, the terminal device 100 receives the results of the additional measurements via the UI screen 600. In other words, the operator inputs the measured values ​​from the additional measurements via the UI screen 600.

[0051] In S307, the terminal device 100 performs a condition diagnosis using the results of the additional measurements received. In this embodiment, the threshold corresponding to the damage mode acquired in S304 and the actual measured value of the additional measurement received in S306 are used to identify the action to be taken. For example, if the actual value is smaller than the threshold, repair is performed, and if the actual value is greater than or equal to the threshold, replacement is performed. Note that the handling of the threshold acquired in S304 may differ depending on the type of part or depending on the damage mode.

[0052] In S308, terminal device 100 performs a condition diagnosis using the acquired measurement results. In this process, no additional measurements are taken, and for example, the damage mode and corresponding content (in this example, "replacement" as the diagnostic target) may be identified as the condition diagnosis results.

[0053] In S309, the terminal device 100 outputs the diagnostic results from S307 or S308 via the UI screen of the UI unit 104. Figure 6 shows an example of the UI screen 600 displayed in this process. Details of the UI screen 600 will be described later. Then, this processing flow ends.

[0054] (UI screen) Figures 4 to 6 illustrate examples of the UI screen configuration displayed in the terminal device 100 according to this embodiment. Figure 4 shows an example of the UI screen configuration displayed during the real-time image diagnostic processing shown in Figure 2. Figures 5 and 6 show examples of the UI screen configuration displayed during the detailed diagnostic processing shown in Figure 3.

[0055] The UI screen 400 shown in Figure 4 displays a real-time image of the inner ring 401 of the rolling mechanism being diagnosed. By inputting the real-time image into the trained model described above, the area of ​​damage and its type (damage mode) are identified. In this example, two areas of damage and their damage mode, "delamination," are identified on the rolling surface of the inner ring 401. Icons 402 and 403, which indicate the information identified on the real-time image, are superimposed on the image.

[0056] As shown in UI screen 400, the results of the condition diagnosis are displayed in real time, allowing the operator to easily understand the general state of the item being diagnosed. Although not shown in Figure 4, instructions and notifications for processes S205 and S211 in Figure 2 may be superimposed on UI screen 400. Furthermore, the shooting instruction for process S208 in Figure 2 may be received using the conventional shooting function provided by the terminal device 100. Additionally, as a type of conventional shooting function, dimensions and zoom / reduction information may be superimposed on the real-time image.

[0057] Next, we will describe the UI screen used in the detailed diagnostic process. The UI screen 500 shown in Figure 5 includes a menu button 501. When the menu button 501 is selected, a selection screen (not shown) for selecting the rolling device to be diagnosed is displayed. The information of the rolling device selected on the selection screen (not shown) is displayed on bar 502. This selection screen may be configured to display a list of products that can be diagnosed and to allow the operator to select one. Furthermore, the information entered on this selection screen may be used in the real-time diagnostic process, for example, as acquired in the process S201 in Figure 2. In this example, bar 502 displays "AAA Company," which indicates the manufacturer of the rolling device, and "XXXXX," which indicates the product information.

[0058] When the camera icon 503 is selected, the system transitions to the UI screen 400 shown in Figure 4. In other words, the system may be configured to perform real-time diagnostic processing when the camera icon 503 is selected. When the folder icon 504 is selected, a selection screen (not shown) for selecting still images that have already been taken is displayed. This selection screen may be configured to allow selection of still images taken during the real-time diagnostic processing.

[0059] Still image 505 shows the still image selected as the image to be diagnosed. In this example, a still image of the inner ring of the rolling gear is selected and displayed. Settings 506, 507, and 508 are parameters for the operator to set detailed information about the item to be diagnosed. In this example, the model number of the rolling gear is entered in setting 506. The type of component that makes up the rolling gear, i.e., the component shown in still image 505, is entered in setting 507. The part of the component to be diagnosed is entered in setting 508. In this example, the model number "XXXXX", component "inner ring", and diagnostic target "rolling surface" are entered. The settings may be configured to be selectable from a list or to be entered manually. The number and types of setting items may also vary depending on the product.

[0060] When the diagnostic button 509 is pressed, the detailed diagnostic process is executed. In other words, the process from S302 onwards in Figure 3 begins. The status icon 510 indicates the current status of the detailed diagnostic process, and here, "Setting," which is the state in which setting input is accepted, may be indicated by being lit up or by other means.

[0061] In the UI screen 600 shown in Figure 6, the menu button 601 is the same as the menu button 501 in the UI screen of Figure 5. The diagnostic result image 602 displays the status information in correspondence with the still image 505 in Figure 5. In this example, two damaged areas and their damage mode "peeling" are identified and shown as icons 603 and 604.

[0062] In this example, we will explain the process assuming that additional measurements are required. Field 605 is an item for inputting the measured values ​​of the additional measurements. When the method button 606 is pressed, the method for the additional measurements is displayed. Specifically, a video 700 as shown in Figure 7 is displayed. The content displayed when the method button 606 is pressed varies depending on the content of the additional measurements. When the measured values ​​are entered into field 605 and the input button 607 is pressed, a condition diagnosis is performed using the measured values ​​of the additional measurements. This corresponds to the process in S307 of Figure 3. The result is then displayed in result 608. If it is determined that additional measurements are not required, field 605, method button 606, and input button 607 may be hidden. If multiple additional measurements are required, multiple fields 605 and method buttons 606 may be displayed corresponding to each additional measurement. If it is determined that there are no abnormalities in the condition diagnosis, result 608 may display a statement to that effect, or it may display "Reuse".

[0063] Response information 609 shows information about actions taken in relation to the results of the condition diagnosis. For example, if "repair" is notified in result 608, information about the effects obtained by actually performing the repair may be displayed. Specifically, the information shown in response information 609 may be environmental information (such as CO2 reduction amount) or cost (difference between repair and replacement).

[0064] When the transition button 610 is pressed, the user is redirected to a website where the repaired parts can be purchased, or to a website of a company that undertakes the replacement work. The destination at this point may be switched based on the product information entered via the UI screen. The status icon 611 indicates the status of the current detailed diagnostic process, and here, "Results," which indicates that the diagnostic results have been output, may be indicated by being lit up or otherwise.

[0065] Note that the UI screen configurations shown in Figures 4 to 6 are examples only and are not limiting. For example, the displayed items may be changed depending on the product being diagnosed.

[0066] Furthermore, in the configuration shown in Figure 3, if it was determined that additional measurements were necessary in step S303, the system would input the actual measured values ​​from the additional measurements before performing a condition diagnosis. However, the system is not limited to this configuration. For example, even if it is determined that additional measurements are necessary, the system could perform a condition diagnosis based on the information acquired at that time and output a simplified diagnosis result. For example, if it is necessary to additionally measure the length of damage using calipers, and the dimensions of the damage can be estimated from acquired still or moving images, a simplified diagnostic result may be output using these estimated dimensions. This allows the operator to understand the current state diagnosis before performing additional measurements. This is useful, for example, when additional measurements would be time-consuming or laborious.

[0067] The correspondence tables 800 and 900 shown in Figures 8 and 9 may be maintained on the terminal device 100 side or on the server device 200 side. The terminal device 100 may query the server device 200 during diagnosis to refer to the information defined in the correspondence tables 800 and 900 in the database. In addition, information related to the results of the status diagnosis, such as video 700, may also be obtained by the terminal device 100 querying the server device 200.

[0068] In summary, this embodiment makes it possible to implement a simple and highly user-friendly condition diagnosis method for machinery and equipment. In particular, even non-experts can easily grasp the condition of the machinery and equipment and take appropriate action smoothly.

[0069] <Other Embodiments> Furthermore, the present invention can also be realized by supplying a program or application for realizing the functions of one or more embodiments described above to a system or device using a network or storage medium, and having one or more processors in the computer of that system or device read and execute the program.

[0070] Alternatively, it may be implemented by a circuit that performs one or more functions (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

[0071] Furthermore, where the terms "first" and "second" are used in this specification, they are used merely for convenience to distinguish them from other elements, and are not intended to be interpreted restrictively as referring to specific elements. Therefore, these expressions should be interpreted appropriately depending on the combination and number of constituent elements.

[0072] Thus, the present invention is not limited to the embodiments described above. It is also intended and within the scope of protection to be provided for the combination of each configuration of the embodiments, as well as for modifications and applications by those skilled in the art based on the description in the specification and well-known technology.

[0073] As described above, the following matters are disclosed in this specification: (1) Acquisition means (e.g., 101, 103) for acquiring images including the object to be diagnosed, A means for identifying the damaged area and type of damage in an image acquired by the acquisition means (for example, 101) is provided, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image. Output means (for example, 101, 102, 104, 400) that outputs the image acquired by the acquisition means and the damaged part and type of damage identified by the identification means in association, A condition diagnostic system (e.g., 100) having the following. This configuration allows for easy implementation and enables a more user-friendly condition diagnosis method for machinery and equipment. In particular, even non-experts can easily understand the condition of the machinery and equipment and take appropriate action smoothly.

[0074] (2) The condition diagnosis system according to (1), further comprising a diagnostic means that identifies a corresponding to the diagnostic target using the damaged portion and type of damage in the image acquired by the acquisition means and outputs it to a user interface screen (e.g., 600). This configuration allows for easy implementation and enables a more user-friendly condition diagnosis method for machinery and equipment. In particular, even non-experts can easily understand the condition of the machinery and equipment and take appropriate action smoothly.

[0075] (3) A determination means (for example, 101) that determines whether additional measurements are necessary for the object to be diagnosed based on the damaged part and the type of damage determined by the identification means, If the determination means determines that additional measurements are necessary, a determination means (for example, 101) determines information for additional measurements to the diagnostic target according to the damaged portion and type of damage identified by the identification means, A display control means (e.g., 101) that displays the information of the additional measurement determined by the determination means and a field for inputting the measured value obtained from the additional measurement on the user interface screen (e.g., 600), It has, The diagnostic means further identifies a corresponding to the diagnostic target using the measured values ​​and outputs it to the user interface screen (e.g., 608), the state diagnostic system according to (2). This configuration allows operators to easily identify the additional measurements that need to be performed on the subject of diagnosis. It also improves the ease of inputting actual measured values.

[0076] (4) The condition diagnostic system according to (3), wherein the information of the additional measurement includes lecture information (e.g., 700) of the additional measurement for the subject of diagnosis. This configuration allows operators to easily understand the methods for any additional measurements required for the machinery.

[0077] (5) The condition diagnostic system according to any one of (2) to (4), wherein the response includes at least one of repair and replacement of the subject of the diagnosis. This configuration makes it easy to determine whether to repair or replace damaged machinery.

[0078] (6) The condition diagnostic system described in (5), further including determining that the above response is for reuse. This configuration indicates whether the machinery is reusable depending on its condition, making it easy for workers to understand that the machinery is reusable.

[0079] (7) The diagnostic means further outputs information on the user interface screen (e.g., 609) when an action is taken with respect to the object to be diagnosed, the condition diagnostic system according to any one of (2) to (6). This configuration makes it easy to understand the effects of taking the suggested measures on damaged machinery.

[0080] (8) The condition diagnostic system according to any one of (1) to (7), wherein the object to be diagnosed is a rolling device. This configuration makes it possible to implement a condition diagnostic method for rolling mechanisms that is easy to install and offers greater usability.

[0081] (9) An acquisition step (for example, S202, S301) in which an image including the object to be diagnosed is acquired, A selection step (for example, S206, S302) identifies the damaged area and type of damage in the image acquired in the acquisition step using a trained model that takes an image as input and outputs the damaged area and type of damage contained in the image to be diagnosed, An output step (for example, S207, S209, S302) outputs the image acquired in the acquisition step and the damaged area and type of damage identified in the identification step in association with each other. Methods for diagnosing a severe condition. This configuration allows for easy implementation and enables a more user-friendly condition diagnosis method for machinery and equipment. In particular, even non-experts can easily understand the condition of the machinery and equipment and take appropriate action smoothly.

[0082] (10) A computer (for example, 100) An acquisition means (e.g., 101) for acquiring an image containing the object to be diagnosed, A means for identifying the damaged area and type of damage in an image acquired by the acquisition means (for example, 101) is provided, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image. An output means (for example, 101) that outputs an image acquired by the acquisition means and the damaged portion and type of damage identified by the identification means in association with each other, A program to execute. This configuration allows for easy implementation and enables a more user-friendly condition diagnosis method for machinery and equipment. In particular, even non-experts can easily understand the condition of the machinery and equipment and take appropriate action smoothly.

[0083] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to these examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. Furthermore, the components of the above embodiments may be combined in any way without departing from the spirit of the invention.

[0084] This application is based on Japanese Patent Application No. 2023-210073 filed on December 13, 2023, and its contents are incorporated herein by reference. [Explanation of Symbols]

[0085] 100…Terminal device 101... Processing Section 102...Storage section 103...Photography Department 104...UI section 105... Communications Department 200…Server device 201... Processing Unit 202...Storage section 203... Communications Department 204…Input / output section 300…Network

Claims

1. A means for acquiring images that include the subject of diagnosis, An identification means that identifies the damaged area and type of damage in an image acquired by the acquisition means, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image, An output means that outputs an image acquired by the acquisition means and the damaged portion and type of damage identified by the identification means, A diagnostic means that identifies a corresponding response to the diagnostic target using the damaged portion and type of damage in the image acquired by the acquisition means and outputs it to the user interface screen, The aforementioned identification means determines whether additional measurements are necessary for the object to be diagnosed based on the damaged portion and the type of damage, If the determination means determines that additional measurements are necessary, the determination means determines information for additional measurements to the diagnostic target according to the damaged portion and type of damage identified by the identification means, A display control means that displays on the user interface screen the information of the additional measurement determined by the determination means and a field for inputting the measured values ​​obtained from the additional measurement, It has, The diagnostic means further includes a state diagnostic system that uses the measured values ​​to identify a corresponding response to the diagnostic target and outputs it to the user interface screen.

2. The condition diagnosis system according to claim 1, wherein the information of the additional measurement includes lecture information on the additional measurement for the subject to diagnosis.

3. The condition diagnosis system according to claim 1, wherein the response includes at least one of repair and replacement of the subject of diagnosis.

4. The status diagnosis system according to claim 3, further comprising determining that the above response is for reuse.

5. The state diagnosis system according to claim 1, wherein the diagnostic means further outputs information on the user interface screen regarding the actions taken for the object to be diagnosed.

6. The condition diagnosis system according to claim 1, wherein the object to be diagnosed is a rolling device.

7. The acquisition process involves obtaining images that include the subject of diagnosis, A selection step involves using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image to be identified in the acquisition step, An output step that outputs the image acquired in the acquisition step, along with the damaged area and type of damage identified in the identification step, A diagnostic step which uses the damaged area and type of damage in the image acquired in the acquisition step to identify the appropriate response for the diagnostic target and output it to the user interface screen, A determination step is performed to determine whether additional measurements are necessary for the object to be diagnosed based on the damaged area and the type of damage in the specified step. If the determination step determines that additional measurements are necessary, a determination step is performed to determine information for additional measurements of the target to be diagnosed, corresponding to the damaged portion and type of damage identified in the identification step. A display control step that displays the information of the additional measurement determined in the determination step and a field for inputting the measured value obtained in the additional measurement on the user interface screen, It has, The diagnostic step further includes a condition diagnosis method that uses the measured values ​​to identify a corresponding response to the diagnostic target and outputs it to the user interface screen.

8. On the computer, A means for acquiring images that include the subject of diagnosis, An identification means that identifies the damaged area and type of damage in an image acquired by the acquisition means, using a trained model that takes an image as input and outputs the damaged area and type of damage to be diagnosed in the image, An output means that outputs an image acquired by the acquisition means and the damaged portion and type of damage identified by the identification means, A diagnostic means that identifies a corresponding response to the diagnostic target using the damaged portion and type of damage in the image acquired by the acquisition means and outputs it to the user interface screen, The aforementioned identification means determines whether additional measurements are necessary for the object to be diagnosed based on the damaged portion and the type of damage, If the determination means determines that additional measurements are necessary, the determination means determines information for additional measurements to the diagnostic target according to the damaged portion and type of damage identified by the identification means, A display control means that displays on the user interface screen the information of the additional measurement determined by the determination means and a field for inputting the measured values ​​obtained from the additional measurement, A program to execute, The diagnostic means further includes a program that uses the measured values ​​to identify a corresponding response to the diagnostic target and outputs it to the user interface screen.

Images