Inspection system and inspection device

The inspection system addresses accuracy issues by inversely transforming corrected images to determine the correction formula, ensuring consistent inspection results across users without requiring individual models, thus improving efficiency and accuracy.

JP2024163520APending Publication Date: 2024-11-22JTEKT CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023079214
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing inspection systems face accuracy issues when using corrected tissue images as training data, as different users may apply varying correction processes, leading to decreased inspection results and increased man-hours in constructing trained models.

Method used

An inspection system that performs inverse transformation of gamma and contrast correction processes on corrected images to determine the correction formula used, allowing for accurate inspection without needing separate models for each user, by comparing brightness values and specifying the correction formula.

Benefits of technology

Enables accurate inspection by appropriately transforming corrected images, maintaining inspection accuracy regardless of user-specific correction methods, reducing the need for multiple user-specific models and man-hours.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024163520000001_ABST
    Figure 2024163520000001_ABST
Patent Text Reader

Abstract

To provide a technology capable of properly, inversely converting a correction image processed by a gamma correction and capable of properly performing inspection based on the correction image.SOLUTION: An inspection system 1 comprises an inspection device 2. The inspection device 2 comprises a processing part 2a to execute a pre-processing performed before an inverse transformation. The pre-processing includes a first processing and a second processing. The first processing obtains a first inverse transformation image processed by the inverse transformation based on a first gamma correction formula with respect to a first correction image processed by the gamma correction and a second inverse transformation image processed by an inverse transformation based on a second gamma correction formula with respect to the first correction image. The second processing identifies whether the gamma correction formula used for the gamma correction processing is the first gamma correction formula or a second gamma correction formula by comparing a first difference of a luminance value between a texture image and the first inverse transformation image with a second difference of the luminance value between the texture image and the second inverse transformation image. The inverse transformation processing includes the first inverse transformation processing based on the identified gamma correction formula.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an inspection system and an inspection apparatus. [Background technology]

[0002] Patent Document 1 discloses an inspection system that inspects the carbon concentration and metal structure based on a cross-sectional structure image of a member made of heat-treated carbon steel or alloy steel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-144402 A Summary of the Invention [Problem to be solved by the invention]

[0004] The structure image of the metal member is acquired by a digital camera or the like attached to a metallurgical microscope. The structure image acquired by the digital camera may be provided to an image processing device. The image processing device has a function for performing contrast correction processing and gamma correction processing on the tissue image. The user uses the functions of the image processing device to perform correction processing on the tissue image that is suitable for visual observation through a microscope, and performs observation using the corrected image.

[0005] Here, the inspection system is configured to perform inspection using a trained model obtained by machine learning using uncorrected tissue images as training data. Therefore, for example, when a corrected image that has been subjected to the correction process is provided to the inspection system, there is a risk that the accuracy of the inspection results will decrease. On the other hand, if a trained model is constructed using corrected images that have been corrected by a user as training data, it is possible to suppress any decrease in accuracy of inspection results that use the corrected images.

[0006] However, the correction contents in the correction process may differ among multiple users or may differ depending on the image processing device. Therefore, when building a trained model using the corrected image as training data, it is necessary to build a trained model for each user, which leads to an increase in labor hours.

[0007] On the other hand, even if each user provides a corrected image, if the correction made to that corrected image can be reverse-converted, the inspection system can obtain an image equivalent to the tissue image before the correction process was performed based on each user's corrected image, and there is no need to construct a trained model for each user.

[0008] In particular, when a user provides the above-mentioned inspection system with a gamma-corrected corrected image, there may be a significant decrease in the accuracy of the inspection results, and a method is desired that can properly inverse-convert the gamma-corrected corrected image to enable proper inspection. [Means for solving the problem]

[0009] The inspection system according to the embodiment is an inspection system for a target member made of heat-treated carbon steel or alloy steel. The inspection system includes an image acquisition unit that acquires a structural image obtained by observing a cross section of the target member under a microscope, an image conversion device that can output one or more corrected images obtained by performing a correction process on the structural image and the structural image, and an inspection device that inspects the target member based on the corrected images. The inspection device includes a processing unit that executes an inverse conversion process that performs an inverse conversion of the correction process on the corrected image, a quality inspection process that performs a quality inspection of the target member based on the inversely converted structural image obtained by the inverse conversion process, and a pre-processing process that is performed before the inverse conversion process. The correction process includes a gamma correction process that is performed using either a first gamma correction formula or a second gamma correction formula that is different from the first gamma correction formula. The corrected image includes a first corrected image that has been subjected to the gamma correction process. The pre-processing includes a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process based on the first gamma correction formula on the first corrected image and a second inverse-transformed image obtained by performing an inverse transformation process based on the second gamma correction formula on the first corrected image, and a second process for identifying whether the gamma correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse-transformed image and a second difference in luminance values ​​between the tissue image and the second inverse-transformed image. The inverse transformation process includes a first inverse transformation process based on the identified gamma correction formula.

[0010] Another embodiment from a different viewpoint is an inspection device for a target member made of heat-treated carbon steel or alloy steel. The inspection device includes a processing unit that executes a process for receiving a corrected image obtained by performing a correction process on a structural image obtained by observing a cross section of the target member under a microscope, the structural image, an inverse transformation process for performing an inverse transformation of the correction process on the corrected image, a quality inspection process for inspecting the quality of the target member based on the inverse transformation structural image obtained by the inverse transformation process, and a pre-processing process that is performed before the inverse transformation process. The correction process includes a gamma correction process that is performed using either a first gamma correction formula or a second gamma correction formula that is different from the first gamma correction formula. The corrected image includes a first corrected image that has been subjected to the gamma correction process. The pre-processing includes a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process based on the first gamma correction formula on the first corrected image and a second inverse-transformed image obtained by performing an inverse transformation process based on the second gamma correction formula on the first corrected image, and a second process for identifying whether the correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse-transformed image and a second difference in luminance values ​​between the tissue image and the second inverse-transformed image. The inverse transformation process includes a first inverse transformation process based on the identified gamma correction formula. Effect of the Invention

[0011] According to the present disclosure, a corrected image that has been subjected to gamma correction processing can be appropriately inverse-converted, and inspection can be appropriately performed based on the corrected image. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example of an inspection system. [Diagram 2] FIG. 2 is a block diagram showing functions of the processing unit. [Diagram 3] FIG. 3 is a diagram showing an example of a sample for observing a cross section of a raceway using a microscope. [Figure 4]FIG. 4 is a flowchart showing an example of the correction process. [Diagram 5] FIG. 5 is a flowchart showing an example of an inspection process performed by the inspection device. [Figure 6] FIG. 6 is a flowchart showing an example of the pre-processing. [Figure 7] FIG. 7 is a diagram showing the flow of various information obtained by processing a group of preprocessing images in preprocessing. [Figure 8] FIG. 8(a) is a diagram showing an example of a tissue image included in the group of preprocessing images, and FIG. 8(b) is a diagram showing an example of a first corrected image included in the group of preprocessing images. [Figure 9] FIG. 9(a) is a diagram showing an example of a first inverse transformed image, and FIG. 9(b) is a diagram showing an example of a second inverse transformed image. [Figure 10] FIG. 10(a) is a diagram showing an example of a second corrected image included in the pre-processing image group, and FIG. 10(b) is a diagram showing an example of a second histogram obtained using the second corrected image shown in FIG. 10(a). [Figure 11] FIG. 11 shows the results of the verification test. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First, the contents of the embodiment will be listed and described. [Outline of the embodiment] (1) An embodiment of an inspection system is an inspection system for a target member made of heat-treated carbon steel or alloy steel. The inspection system includes an image acquisition unit that acquires a structural image obtained by observing a cross section of the target member under a microscope, an image conversion device that can output one or more corrected images obtained by performing a correction process on the structural image and the structural image, and an inspection device that inspects the target member based on the corrected images. The inspection device includes a processing unit that executes an inverse conversion process that performs an inverse conversion of the correction process on the corrected image, a quality inspection process that performs a quality inspection of the target member based on the inverse converted structural image obtained by the inverse conversion process, and a pre-processing process that is performed before the inverse conversion process. The correction process includes a gamma correction process that is performed using either a first gamma correction formula or a second gamma correction formula different from the first gamma correction formula. The corrected image includes a first corrected image that has been subjected to the gamma correction process. The pre-processing includes a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process based on the first gamma correction formula on the first corrected image and a second inverse-transformed image obtained by performing an inverse transformation process based on the second gamma correction formula on the first corrected image, and a second process for identifying whether the gamma correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse-transformed image and a second difference in luminance values ​​between the tissue image and the second inverse-transformed image. The inverse transformation process includes a first inverse transformation process based on the identified gamma correction formula.

[0014] According to the above configuration, the gamma correction formula used in the gamma correction process can be identified by the first process and the second process. Therefore, even if it is unknown whether the first gamma correction formula or the second gamma correction formula was used in the gamma correction process, appropriate inverse conversion (first inverse conversion process) can be performed based on the identified gamma correction formula. As a result, inspection can be performed based on the corrected image.

[0015] (2) In the above-mentioned inspection system, when the correction process further includes a contrast correction process, the plurality of corrected images include a second corrected image subjected to the contrast correction process, and the pre-processing further includes a third process of calculating a value of a contrast correction amount included in a contrast correction formula used in the contrast correction process based on the tissue image, the second corrected image, and the contrast correction formula, the inverse conversion process may further include a second inverse conversion process based on the contrast correction formula in which the contrast correction amount is the value. In this case, even if the correction process includes a contrast correction process, the inverse conversion can be performed appropriately.

[0016] (3) In the above inspection system, when the plurality of corrected images include a third corrected image corrected by both the gamma correction process and the contrast correction process, the preprocessing may further include a fourth process of obtaining a third inverse-conversion image obtained by performing inverse conversion processing on the third corrected image in the order of the first inverse conversion process and the second inverse conversion process, and a fourth inverse-conversion image obtained by performing inverse conversion processing on the third corrected image in the order of the second inverse conversion process and the first inverse conversion process, and a fifth process of identifying an order of the first inverse conversion process and the second inverse conversion process in the inverse conversion process by comparing a third difference in luminance values ​​between the tissue image and the third inverse-conversion image and a fourth difference in luminance values ​​between the tissue image and the fourth inverse-conversion image, and the inverse conversion process may be configured to perform the first inverse conversion process and the second inverse conversion process in accordance with the order identified in the fifth process. In this case, even if the order of the gamma correction process and the contrast correction process is unknown in the correction process, the fifth process can determine the order of the first inverse conversion process and the second inverse conversion process, and even if the correction process includes both the gamma correction process and the contrast correction process, the inverse conversion can be performed more appropriately by taking into account the order.

[0017] (4) Also, an embodiment from another viewpoint is an inspection device for a target member made of heat-treated carbon steel or alloy steel. This inspection device includes a processing unit that executes a process for receiving a corrected image obtained by performing a correction process on a structural image obtained by observing a cross section of the target member under a microscope, the structural image, an inverse transformation process for performing an inverse transformation of the correction process on the corrected image, a quality inspection process for inspecting the quality of the target member based on the inverse transformed structural image obtained by the inverse transformation process, and a pre-processing process that is performed before the inverse transformation process. The correction process includes a gamma correction process that is performed using either a first gamma correction formula or a second gamma correction formula that is different from the first gamma correction formula. The corrected image includes a first corrected image that has been subjected to the gamma correction process. The pre-processing includes a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process based on the first gamma correction formula on the first corrected image and a second inverse-transformed image obtained by performing an inverse transformation process based on the second gamma correction formula on the first corrected image, and a second process for identifying whether the correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse-transformed image and a second difference in luminance values ​​between the tissue image and the second inverse-transformed image. The inverse transformation process includes a first inverse transformation process based on the identified gamma correction formula.

[0018] [Details of the embodiment] Hereinafter, preferred embodiments will be described with reference to the drawings. [Overall system configuration] FIG. 1 is a block diagram illustrating an example of an inspection system. An inspection system 1 in FIG. 1 is a system for inspecting the quality of a heat-treated product based on a microscopic image of the cross-sectional structure of the heat-treated product. More specifically, the inspection system 1 has a function of inspecting the quality of the heat-treated product based on the cross-sectional structure of the heat-treated product. The quality of the heat-treated product that can be inspected by the inspection system 1 includes at least an estimated value of the carbon concentration of the heat-treated product, an estimated value of the amount of retained austenite of the heat-treated product, and an estimated value of the cross-sectional hardness of the heat-treated product. The inspection system 1 includes an inspection device 2 and a plurality of user side devices 4 .

[0019] The multiple user-side devices 4 are installed, for example, in multiple factories scattered around the country. Each of the multiple user-side devices 4 is used by an inspector (user) who inspects the heat-treated product. Each of the multiple user side devices 4 includes a user terminal 6, a camera 8, and a microscope 10. The microscope 10 is a metallurgical microscope capable of observing a cross section of a metal. The camera 8 is incorporated into the microscope 10. The camera 8 is an imaging device having an imaging element such as a CCD element or a CMOS element, and an optical mechanism such as a lens. The camera 8 has a function of capturing an image of the metal structure observed by the microscope and acquiring a structure image. The camera 8 is connected to the user terminal 6 and provides the user terminal 6 with the acquired structural image of the metal structure.

[0020] The user terminal 6 is, for example, a computer, and includes a processing unit 6a, a storage unit 6b, and an input / output unit 6c. The processing unit 6a is composed of, for example, a CPU (Central Processing Unit). The storage unit 6b is, for example, a flash memory, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The storage unit 6b stores computer programs to be executed by the processing unit 6a and necessary information. The processing unit 6a realizes various processing functions possessed by the processing unit 6a by executing computer programs stored in a computer-readable non-transitory recording medium such as the storage unit 6b. The input / output unit 6c has a function of receiving input from a user and outputting information, etc. to the user. The input / output unit 6c is, for example, a keyboard, a mouse, a touch panel, a monitor, a speaker, a printer, an indicator, etc.

[0021] The user terminal 6 and the inspection device 2 are connected to each other so as to be able to communicate with each other via a network NW such as a LAN (Local Area Network) or the Internet. Therefore, the user terminal 6 can transmit the tissue image and the corrected image (described later) provided by the camera 8 to the inspection device 2. Furthermore, the user terminal 6 can receive information such as the test results transmitted from the testing device 2. The user terminal 6 can output the information such as the test results to the user via the input / output unit 6c.

[0022] The inspection device 2 is, for example, a computer such as a server. The inspection device 2 may be provided on a cloud. The inspection device 2 has a function of performing quality inspection processing of the heat-treated product based on images transmitted from a plurality of user terminals 6.

[0023] The inspection device 2 includes a processing unit 2a and a storage unit 2b. The processing unit 2a is composed of, for example, a CPU (Central Processing Unit). The storage unit 2b is, for example, a flash memory, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The storage unit 2b stores computer programs to be executed by the processing unit 2a and necessary information. The processing unit 2a executes computer programs stored in a computer-readable non-transitory recording medium such as the storage unit 2b to realize various processing functions possessed by the processing unit 2a.

[0024] The memory unit 2b also stores a trained model 12 and a setting table 14. The trained model 12 is a model that has been trained to output a value indicating the quality of a sample when given features obtained from a tissue image of the sample to be inspected in the inspection process described below. In the setting table 14, setting information required for the inverse conversion process (described later) is registered.

[0025] FIG. 2 is a block diagram showing the functions of the processing unit 2a. The processing unit 2a executes a computer program stored in the storage unit 2b to perform a pre-processing 2a1, an inverse conversion process 2a2, and a quality inspection process 2a3. Each of these processes will be described later.

[0026] [Regarding the subject material to be inspected and the microscopic observation of the subject material] The inspection system 1 can inspect a target member. The target member includes a heat-treated product using carbon steel, alloy steel, etc. Examples of carbon steel include S10C, S25C, S40C, S45C, etc. Examples of alloy steel include SCr415, SCM415, SCM435, SNCM420, SAE5120, etc. Furthermore, the heat treatment of the target member includes carburizing and quenching.

[0027] In this embodiment, the subject member is a raceway of a rolling bearing. The raceway is a member made of carbon steel or alloy steel. The raceway is carburized and quenched, and then tempered. The carburizing, quenching and tempering are performed for each lot including a plurality of races. One or more of the races included in the lot are selected as the target member.

[0028] FIG. 3 is a diagram showing an example of a sample for observing a cross section of a raceway using a microscope. In FIG. 3, the sample 100 includes a cut piece 102 and a resin 104 . The cut piece 102 is a part of the raceway that has been cut out using a high-speed cutter or the like. The resin 104 is a member formed in a cylindrical shape, and holds the cut pieces 102 by embedding the cut pieces 102 therein. The observation surface 100a of the sample 100 includes a cross section 102a of the cut piece 102 and a resin surface 104a. Therefore, the cross section 102a is exposed on the observation surface 100a of the sample 100. The cross section 102a is a cross section along a plane including the axis of the raceway ring, and therefore the raceway surface 102a1 appears in the cross section 102a.

[0029] The observation surface 100a is mirror-polished and then immersed in an etching solution for observing a structure, such as nital (a nitric acid alcohol solution), for a predetermined time, whereby the cross section 102a is corroded (etched). The bearing ring selected as the target member is machined by the user into a cut piece 102 and used to prepare a sample 100.

[0030] The sample 100 with the etched cross section 102a is set in the microscope 10. The user sets a required portion of the cross section 102a in the field of view of the microscope 10 and captures a tissue image. In this inspection process, the carburized layer of the raceway is the object of inspection. Therefore, the user places the portion of the carburized layer in the cross section 102a in the field of view of the microscope 10 and captures a structural image with the camera 8. As a result, a structural image of the carburized layer in the cross section 102a is captured.

[0031] The carburized layer is present in a range from raceway surface 102a1 on cross section 102a to a depth of about 1 mm to 3 mm directly below raceway surface 102a1. Therefore, the user captures an image of the cross section region at a depth position (predetermined depth position) of 1 mm to 3 mm from raceway surface 102a1 (surface) using camera 8 to obtain a structure image. In this way, the camera 8 constitutes an image acquisition unit that acquires tissue images obtained by observing a cross section of a target member under a microscope. The tissue image captured by the camera 8 is provided to the user terminal 6.

[0032] [Regarding correction processing by user terminal 6] The user terminal 6 has a function of performing a correction process on the tissue image. The user terminal 6 obtains a corrected image by performing the correction process on the tissue image. The correction process includes a gamma correction process and a contrast correction process. The correction process is realized by executing an image conversion program stored in the storage unit 6b of the user terminal 6 by the processing unit 6a. In this manner, the corrected image is an image obtained by performing correction processing on the tissue image. Note that in this embodiment, the corrected image refers to an image that has been subjected to at least one of gamma correction processing and contrast correction processing. As described above, the tissue image is an image provided by the camera 8 and is an image before correction processing is performed. In this embodiment, the tissue image refers to an image that has not been subjected to either gamma correction processing or contrast correction processing.

[0033] FIG. 4 is a flowchart showing an example of the correction process. The correction process performed by the user terminal 6A, out of the multiple user terminals 6 in FIG. 1, will be described below. The processing unit 6a of the user terminal 6A first determines whether or not a tissue image has been provided from the camera 8 (step S21 in FIG. 4). The processing unit 6a repeats step S21 until it determines that a tissue image has been received.

[0034] When it is determined that the tissue image has been received, the processing unit 6a performs a gamma correction process on the tissue image (step S22 in FIG. 4). Here, the gamma correction process is performed using either the following equation (1) or equation (2).

[0035]

number

[0036] In the above formulas (1) and (2), (x, y) are the coordinates of each pixel in the tissue image and the image after the correction process, and indicate the X-direction coordinate and Y-direction coordinate of each pixel when the X-direction and Y-direction are set in the tissue image. P''(x,y) is the luminance value of each pixel contained in the image after gamma correction processing. P(x, y) is the luminance value of each pixel contained in the image before gamma correction processing. γ is a gamma correction value that is set by the user.

[0037] In the first gamma correction formula shown in formula (1), the inverse of the gamma correction value γ is used as the exponent, and in the second gamma correction formula shown in formula (2), the gamma correction value γ is used as the exponent. In this respect, the first gamma correction formula and the second gamma correction formula are different from each other.

[0038] The processing unit 6a of the user terminal 6A performs gamma correction processing using the first gamma correction equation (equation (1)). The processing unit 6a performs gamma correction processing on the tissue image to obtain an intermediate corrected image. It should be noted that the user terminals 6 other than the user terminal 6A also include those that perform gamma correction processing using the second gamma correction formula.

[0039] Next, the processing unit 6a performs contrast correction processing on the intermediate corrected image to obtain a final corrected image (step S23 in FIG. 4), and then ends the processing. The contrast correction process is performed using the following formula (3). P m '(x,y)=α m (P m (x,y)-127)+127+β m (3)

[0040] In the above formula (3), (x, y) are the coordinates of each pixel in the tissue image. P m '(x, y) is the luminance value of each pixel contained in the image after contrast correction processing. P m (x, y) is the luminance value of each pixel contained in the image before the contrast correction process. α m is the contrast correction amount, and β m is the brightness correction amount. In the correction process, the user terminal 6 uses the correction amount α m , β m The user terminal 6 then inputs a correction amount αm , β m For this reason, the user performs contrast correction processing using the correction amount α m , β m The specific value of is not known.

[0041] By completing the correction process, the processing unit 6a can obtain a final corrected image that has been subjected to gamma correction and contrast correction. When the quality inspection process is performed, the processing unit 6a transmits the final corrected image to the inspection device 2.

[0042] Moreover, the processing unit 6a has a function of obtaining a first corrected image in which only gamma correction processing has been performed on the tissue image, and a second corrected image in which only contrast correction processing has been performed on the tissue image. Furthermore, the processing unit 6a has a function of transmitting a first corrected image, a second corrected image, and a final corrected image based on one tissue image together with the one tissue image to the inspection device 2 in response to a request from the inspection device 2. When transmitting these images to the inspection device 2, the processing unit 2a also transmits to the inspection device 2 the gamma correction value γ used when obtaining the first corrected image. The gamma correction value γ when obtaining the first corrected image is the same as the gamma correction value γ when obtaining the intermediate corrected image. m , β m The relative values ​​shown are the same as the relative values ​​when the final corrected image is obtained.

[0043] In this way, the user terminal 6 (the processing unit 6a) constitutes an image conversion device capable of outputting one or more corrected images obtained by performing correction processing on a tissue image, and the tissue image. The correction process is realized by executing an image conversion program. The user terminal 6 that performs the correction process receives the tissue image and necessary parameters (gamma correction value γ and correction amount α m , β mThe image conversion program is configured to output a final corrected image when a gamma correction formula used in the gamma correction process and the order of the gamma correction process and the contrast correction process in the correction process are given.

[0044] For this reason, the user, user terminal 6, and inspection device 2 are unclear about the gamma correction formula used in the gamma correction process and the order of the gamma correction process and the contrast correction process in the correction process. As described above, the correction amount α m , β m Also, this is unknown to the user, the user terminal 6, and the inspection device 2.

[0045] [Regarding inspection processing] FIG. 5 is a flowchart showing an example of the inspection process performed by the inspection device 2. In addition, here, a case where the inspection process is performed based on an image transmitted from the user terminal 6A among the multiple user terminals 6 will be described.

[0046] In the inspection process, the processing unit 2a of the inspection device 2 first performs pre-processing (step S1 in FIG. 5: pre-processing 2a1 in FIG. 2). The pre-processing is a process that is performed before the inverse conversion process, which will be described later. In the pre-processing, setting information required for the inverse conversion process is obtained. The pre-processing will be described later.

[0047] After the pre-processing, the processing unit 2a judges whether or not the final corrected images transmitted from the multiple user side devices 4 (user terminals 6) have been received (step S2 in FIG. 5). The processing unit 2a repeats step S2 until it judges that the final corrected images have been received.

[0048] When it is determined that the final corrected image has been received from the user terminal 6 (6A), the processing unit 2a performs an inverse conversion process (step S3 in FIG. 5: inverse conversion process 2a2 in FIG. 2). The inverse transformation process is a process of performing the inverse transformation of the correction process on the final corrected image. The inverse conversion process includes a first inverse conversion process (step S31 in FIG. 5) and a second inverse conversion process (step S32 in FIG. 5). The first inverse conversion process is a process that performs an inverse conversion of the gamma correction process. The second inverse conversion process is a process that performs an inverse conversion of the contrast correction process. The order of the first inverse transform process and the second inverse transform process is determined by the pre-processing. The inverse transformation process is performed using the information obtained in the pre-processing process.

[0049] The processing unit 2a performs an inverse transformation process on the final corrected image to obtain an inverse transformed tissue image. After completing the inverse conversion process, the processing unit 2a performs a quality inspection process (step S4 in FIG. 5: quality inspection process 2a3 in FIG. 2).

[0050] In the quality inspection process, the processing unit 2a uses the inverse transformed tissue image, obtains a predetermined feature amount based on the inverse transformed tissue image, and outputs a value indicating the quality of the sample based on the feature amount. The processing unit 2a provides the predetermined feature amount to the trained model 12 (FIG. 1) and obtains a value indicating the quality of the sample. The trained model 12 is a model obtained by machine learning the relationship between a predetermined feature amount obtained from a tissue image and a value indicating quality. In other words, the trained model 12 is a model that uses a predetermined feature amount as an input and a value indicating quality as an output.

[0051] The trained model 12 includes a model for obtaining an estimated value of the carbon concentration, a model for obtaining an estimated value of the amount of retained austenite, and a model for obtaining an estimated value of the cross-sectional hardness. The processing unit 2a uses the trained model 12 to obtain an estimated value of the carbon concentration, an estimated value of the amount of retained austenite, and an estimated value of the cross-sectional hardness.

[0052] Here, the training data used in the machine learning of the trained model 12 is collected using a raceway (a raceway for data collection) similar to the raceway that is the target component. The training data includes a predetermined feature amount obtained from the data collection raceway and a value indicating the quality of the data collection raceway from which the feature amount was obtained. The feature amounts contained in the training data are obtained based on a structural image obtained from a sample prepared by cutting a piece from a data collection race.

[0053] Therefore, the specified features provided to the trained model 12 in the quality inspection process should be obtained from the tissue image. In the quality inspection process of this embodiment, predetermined features obtained from the inversely transformed tissue image are provided to the trained model 12. In this embodiment, as described below, the final corrected image can be appropriately inversely transformed, so that even if a specified feature value obtained from the inversely transformed tissue image is provided to the trained model 12, a value indicating quality can be obtained with the same accuracy as when a specified feature value obtained from the tissue image is provided to the trained model 12.

[0054] When the quality inspection process is completed, the processing unit 2a returns to step S2. The processing unit 2a already has the setting information necessary for the inverse conversion process on the final corrected image of the user terminal 6A. Therefore, the processing unit 2a returns to step S2 and waits for the final corrected image to be provided from the user terminal 6A. In this manner, the processing unit 2a executes the inspection process.

[0055] [Pretreatment] As described above, the pre-processing is a process for acquiring setting information required for the inverse conversion process. FIG. 6 is a flowchart showing an example of the pre-processing. The processing unit 2a first requests the user terminal 6A to transmit a tissue image, as well as a first corrected image, a second corrected image, and a final corrected image based on the tissue image (step S11 in FIG. 6). The images requested in step S11 are images for preprocessing. In the following description, the tissue image, as well as the first corrected image, the second corrected image, and the final corrected image based on the tissue image are referred to as a preprocessing image group. In the preprocessing, the final corrected image is also referred to as a third corrected image.

[0056] Next, the processing unit 2a proceeds to step S12 in Fig. 6 and determines whether or not a group of images for preprocessing has been received. The processing unit 2a repeats step S12 until a group of images for preprocessing has been received. If it is determined that the group of images for preprocessing has been received, the processing unit 2a proceeds to step S13 in FIG. 6, and acquires a first inverse-transformed image and a second inverse-transformed image (step S13 in FIG. 6: first processing).

[0057] The first inverse transformed image and the second inverse transformed image are images obtained by performing inverse transformation of the gamma correction on the first corrected image in the preprocessing image group. The first inverse-transformed image is an image obtained by performing inverse transformation on the first corrected image based on the first gamma correction formula (the above formula (1)). The second inverse-transformed image is an image obtained by performing inverse transformation on the first corrected image based on the second gamma correction formula (the above formula (2)).

[0058] Note that the inverse transformation based on the first gamma correction formula (second gamma correction formula) is a process in which the luminance value of each pixel contained in the image that has been subjected to gamma correction processing is substituted for P''(x, y) in the above formula (1) (formula (2)), and P(x, y) is obtained as the luminance value of each pixel contained in the image after inverse transformation. The gamma correction value γ is provided from the user terminal 6A together with the preprocessing image group.

[0059] FIG. 7 is a diagram showing the flow of various information obtained by processing a group of preprocessing images in preprocessing. The processing unit 2a performs an inverse conversion process on the first corrected image using the first gamma correction formula and the second gamma correction formula, to obtain a first inverse converted image and a second inverse converted image.

[0060] Next, the processing unit 2a proceeds to step S14 in FIG. 6, and identifies the gamma correction formula used in the gamma correction process based on a comparison between the first difference and the second difference (step S14 in FIG. 6: second process). This gamma correction process is included in the correction process by user terminal 6A, and as described above, it is unknown to the user, user terminal 6, and inspection device 2 whether the first gamma correction formula or the second gamma correction formula was used in the gamma correction process. Therefore, the processing unit 2a identifies the gamma correction formula used in the gamma correction process based on a comparison between the first difference and the second difference.

[0061] 7, the first difference is a value obtained by using the first inverse transformed image, and the second difference is a value obtained by using the second inverse transformed image. The first difference is a difference between a luminance value of a tissue image in the preprocessing image group and a luminance value of the first inverse-transformed image, that is, the first difference includes a plurality of differences between a plurality of luminance values ​​of a plurality of pixels included in the tissue image and a plurality of luminance values ​​of a plurality of pixels included in the first inverse-transformed image. The second difference is a difference between the luminance value of the tissue image in the preprocessing image group and the luminance value of the second inverse-transformed image, that is, the second difference includes a plurality of differences between a plurality of luminance values ​​of a plurality of pixels included in the tissue image and a plurality of luminance values ​​of a plurality of pixels included in the second inverse-transformed image.

[0062] By comparing the first difference and the second difference, it can be said that the inverse transformed image corresponding to the smaller difference value is more approximate to the tissue image. Therefore, the processing unit 2a compares the sum of the multiple differences included in the first difference with the sum of the multiple differences included in the second difference, and identifies the inversely transformed image corresponding to the difference with the smaller sum. As shown in FIG. 7, as a result of comparing the first difference with the second difference, the processing unit 2a identifies the gamma correction formula used for the identified inversely transformed image as the gamma correction formula used in the gamma correction process.

[0063] The processing unit 2a uses the identified gamma correction formula in a first inverse conversion process of the inverse conversion processes. In this embodiment, the processing unit 6a of the user terminal 6A performs the gamma correction process using the first gamma correction formula as described above. Therefore, the processing unit 2a specifies the first gamma correction formula as the gamma correction formula used in the gamma correction process. Therefore, in the first inverse conversion process, the processing unit 2a performs the inverse conversion of the gamma correction process based on the first gamma correction formula.

[0064] FIG. 8(a) is a diagram showing an example of a tissue image included in the group of preprocessing images, and FIG. 8(b) is a diagram showing an example of a first corrected image included in the group of preprocessing images. FIG. 9(a) is a diagram showing an example of a first inverse transformed image, and FIG. 9(b) is a diagram showing an example of a second inverse transformed image. The processing unit 2a obtains the first inverse-transformed image and the second inverse-transformed image shown in FIG. 9 using the first corrected image shown in FIG. 8(b), and determines a first difference which is the difference between the luminance value of the tissue image shown in FIG. 8(a) and the luminance value of the first inverse-transformed image, and a second difference which is the difference between the luminance value of the tissue image and the luminance value of the second inverse-transformed image, and identifies the gamma correction formula used in the gamma correction process.

[0065] As shown in FIG. 6, when the gamma correction formula is specified, the processing unit 2a proceeds to step S15 in FIG. 6 and calculates the contrast correction amount α m (Step S15 in FIG. 6: third process). As shown in FIG. 7, the processing unit 2a calculates a contrast correction amount α m To determine , a first histogram of intensity values ​​is generated using the tissue image and a second histogram of intensity values ​​is generated using the second corrected image.

[0066] FIG. 10(a) is a diagram showing an example of a second corrected image included in the pre-processing image group, and FIG. 10(b) is a diagram showing an example of a second histogram obtained using the second corrected image shown in FIG. 10(a). The second histogram shown in FIG. 10(b) is a histogram in which the luminance values ​​of a plurality of pixels included in the second corrected image are represented as classes (horizontal axis) and the number of pixels in each class is represented as a frequency (vertical axis). The processing unit 2a acquires the maximum value Pmax and the minimum value Pmin from among the luminance values ​​of a plurality of pixels included in the second corrected image, based on the second histogram. The processing unit 2a uses the tissue image to generate a first histogram similar to the second histogram shown in FIG. 10(b), and obtains the maximum value Pmax and the minimum value Pmin in the tissue image.

[0067] In this embodiment, a histogram is used to obtain the maximum value Pmax and minimum value Pmin in the tissue image and the maximum value Pmax and minimum value Pmin in the second corrected image, but if these maximum value Pmax and minimum value Pmin can be obtained by another method, that method may be used.

[0068] Next, the processing unit 2a calculates the contrast correction amount α m Calculate the following. More specifically, the processing unit 2a calculates a contrast correction amount α based on an equation obtained by substituting the maximum value Pmax of the tissue image and the maximum value Pmax of the second corrected image into the above equation (3) and an equation obtained by substituting the minimum value Pmin of the tissue image and the minimum value Pmin of the second corrected image into the above equation (3). m and brightness correction amount β m The value of can be calculated.

[0069] The processing unit 2a calculates the correction amount α m , β m In other words, the processing unit 2a uses the calculated correction amount α m , β m The value of is substituted into the contrast correction formula (formula (3)), and the inverse transformation of the contrast correction process is performed based on the substituted contrast correction formula.

[0070] As shown in FIG. 6, the contrast correction amount α mAfter determining , the processing unit 2a proceeds to step S16, and acquires the third inverse transformed image and the fourth inverse transformed image (step S16 in FIG. 6: fourth process). The third inverse transformed image and the fourth inverse transformed image are images obtained by performing inverse transformation of the gamma correction and inverse transformation of the contrast correction on the third corrected image in the group of preprocessing images. The inverse conversion of the gamma correction in step S16 is an inverse conversion process based on the gamma correction formula identified in step S14. That is, the inverse conversion of the gamma correction here is a first inverse conversion process. In addition, the inverse transformation of the contrast correction in step S16 is performed using a correction amount α m , β m is the inverse conversion based on the contrast correction formula in which the value is calculated in step S15. That is, the inverse conversion of the contrast correction here is the second inverse conversion process. In addition, the inverse transformation based on the contrast correction formula is a process of substituting the luminance value of each pixel contained in the image that has been subjected to the contrast correction process into P''(x, y) in the above formula (3) and determining P(x, y) as the luminance value of each pixel contained in the image after the inverse transformation.

[0071] The third inverse transformed image is an image obtained by performing inverse transformation on the third corrected image in the order of the first inverse transformation process and the second inverse transformation process. The fourth inverse transformed image is an image obtained by performing inverse transformation on the third corrected image in the order of the second inverse transformation process and the first inverse transformation process.

[0072] As shown in FIG. 6, when the third inverse-transformed image and the fourth inverse-transformed image are acquired, the processing unit 2a proceeds to step S17 in FIG. 6, and based on a comparison between the third difference and the fourth difference, determines the order of the first conversion process and the second conversion process in the inverse conversion process, which is step S3 in FIG. 5 (step S17: fifth process in FIG. 6).

[0073] 7, the third difference is a value obtained by using the third inverse transformed image, and the fourth difference is a value obtained by using the fourth inverse transformed image. The third difference is a difference between the luminance value of the third corrected image in the preprocessing image group and the luminance value of the third inverse transformed image, that is, the third difference includes a plurality of differences between a plurality of luminance values ​​of a plurality of pixels included in the third corrected image and a plurality of luminance values ​​of a plurality of pixels included in the third inverse transformed image. The fourth difference is a difference between the luminance value of the third corrected image in the preprocessing image group and the luminance value of the fourth inverse transformed image, that is, the fourth difference includes a plurality of differences between a plurality of luminance values ​​of a plurality of pixels included in the third corrected image and a plurality of luminance values ​​of a plurality of pixels included in the fourth inverse transformed image.

[0074] By comparing the third difference and the fourth difference, it can be said that the inverse transformed image corresponding to the smaller difference value is more approximate to the third corrected image. Therefore, the processing unit 2a compares the sum of the multiple differences included in the third difference with the sum of the multiple differences included in the fourth difference, and identifies the inversely transformed image corresponding to the difference with the smaller sum. As shown in FIG. 7, as a result of comparing the third difference with the fourth difference, the processing unit 2a determines the order of inverse transformation in the specified inversely transformed image as the order of the first transformation process and the second transformation process in the inverse transformation process. In this embodiment, the processing unit 6a of the user terminal 6A performs the contrast correction process after the gamma correction process as described above. Therefore, the processing unit 2a determines that the fourth inverse-transformed image is closer to the third corrected image than the third inverse-transformed image. Therefore, the processing unit 2a specifies the order of the processes in the inverse transformation process as the second inverse transformation process and the first inverse transformation process.

[0075] As described above, in the pre-processing, the processing unit 2a determines the gamma correction formula (step S14) and the correction amount α m , β m (step S15), and the order of the first inverse transform process and the second inverse transform process in the inverse transform process (step S17) are acquired as information. As described above, the gamma correction formula specified in step S14 is used in the first inverse conversion process. m , βm The value of is used in the second inverse transform process. The order of the first inverse transform process and the second inverse transform process in step S17 is also used in the inverse transform process.

[0076] This information is the setting information required for the inverse conversion process. The processor 2 a registers the setting information in the setting table 14 . User IDs and setting information are associated and registered in the setting table 14. The user ID is identification information assigned to each of the multiple user terminals 6. During preprocessing, the processing unit 2a also requests transmission of the user ID of the user terminal 6 along with the group of preprocessing images, and registers the obtained setting information and the user ID in the setting table 14 in association with each other.

[0077] In this embodiment, the case where processing related to the setting table 14 is omitted in the inspection process shown in Figure 5 is shown, but the inspection process may be configured to have the processing unit 2a determine whether or not the setting information of the user terminal 6 that sent the final corrected image is registered in the setting table 14 when the final corrected image is provided from one of the multiple user terminals 6. In this case, if the setting information is registered in the setting table 14, the processor 2a performs the inverse conversion process and the quality inspection process (steps S3 and S4 in FIG. 5) on the final corrected image. On the other hand, if the setting information is not registered in the setting table 14, the processing unit 2a is made to execute pre-processing for the user terminal 6 (step S1 in FIG. 5). This allows the processing unit 2a to refer to the setting table 14 and appropriately perform inspection based on images from a plurality of users.

[0078] When the processing order in the inverse conversion process is specified in step S17 in FIG. 6, the processing unit 2a ends the pre-processing and returns to step S2 in FIG. After returning to step S2, the processing unit 2a receives the final corrected image from the user terminal 6A, and then performs inverse conversion processing in the order of the second inverse conversion processing and the first inverse conversion processing (step S3 in FIG. 5), and then performs quality inspection processing (step S4 in FIG. 5).

[0079] According to the above configuration, the gamma correction formula used in the gamma correction process in the user terminal 6A can be identified, so that even if it is unknown whether the first gamma correction formula or the second gamma correction formula was used in the gamma correction process, appropriate inverse conversion (first inverse conversion process) can be performed based on the identified gamma correction formula. Furthermore, according to the above configuration, the correction amount α m , β m Since the value of can be calculated, the contrast correction process included in the correction process can also be appropriately inversely converted. As a result, the inspection can be carried out appropriately based on the final corrected image from the user terminal 6A.

[0080] Furthermore, according to the above configuration, even if the order of the gamma correction process and the contrast correction process is unknown in the correction process of the user terminal 6A, the order of the first inverse conversion process and the second inverse conversion process can be determined, and even if the correction process includes both the gamma correction process and the contrast correction process, the inverse conversion can be performed more appropriately by taking into account the order.

[0081] [About verification testing] Next, a verification test performed on the inspection system 1 will be described. The test method consisted of preparing three images: a tissue image, a corrected image (first corrected image) obtained by performing gamma correction on this tissue image, and an inverse-converted tissue image obtained by performing inverse conversion on this corrected image. The three images were then used to obtain an estimate of the carbon concentration.

[0082] FIG. 11 shows the results of the verification test. As shown in the figure, two values ​​(0.8, 1.2) were set for the gamma correction value γ in the gamma correction process, and the estimated carbon concentration was obtained for each of the two gamma correction values ​​γ. As shown in Figure 11, the carbon concentration estimated using tissue images is 0.707%. In contrast, the estimated carbon concentration using the first corrected image was 0.7111% and 0.709%. Thus, it can be seen that the estimated carbon concentration using the image that has been gamma corrected contains an error compared to the estimated carbon concentration using the tissue image.

[0083] The estimated carbon concentration using the inverse transformed tissue images was 0.707% for both gamma correction values ​​γ, which indicates that when the inverse transformed tissue images are also used, a quality value (estimated carbon concentration) equivalent to that of the tissue images can be obtained. From these results, it is clear that the corrected image can be appropriately inverse transformed, and that inspection can be appropriately performed based on the corrected image.

[0084] 〔others〕 The embodiments disclosed herein are illustrative in all respects and are not restrictive. The scope of the present invention is not limited to the above-described embodiment, but includes all modifications within the scope of the equivalents of the configurations described in the claims. [Explanation of symbols]

[0085] 1 Inspection system 2. Inspection equipment 2a Processing section 6 User terminal (image processing device) 8 Camera (image acquisition unit) 10. Microscope

Claims

1. 1. A system for inspecting a target component made of heat treated carbon or alloy steel, comprising: an image acquisition unit for acquiring a structure image obtained by observing a cross section of the target member under a microscope; an image conversion device capable of outputting one or more corrected images obtained by performing a correction process on the tissue image and the tissue image; an inspection device that inspects the target component based on the corrected image, The inspection device includes: an inverse transformation process for performing an inverse transformation of the correction process on the corrected image; a quality inspection process for inspecting the quality of the target component based on the inverse transformation tissue image obtained by the inverse transformation process; A pre-processing step performed before the inverse transformation process; a processing unit for executing the correction process includes a gamma correction process performed using either a first gamma correction formula or a second gamma correction formula different from the first gamma correction formula, the corrected image includes a first corrected image that has been subjected to the gamma correction process, The pretreatment is a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process on the first corrected image based on the first gamma correction formula, and a second inverse-transformed image obtained by performing an inverse transformation process on the first corrected image based on the second gamma correction formula; a second process of identifying whether a gamma correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse transformed image and a second difference in luminance values ​​between the tissue image and the second inverse transformed image, The inverse conversion process includes a first inverse conversion process based on the specified gamma correction formula. Inspection system.

2. The correction process further includes a contrast correction process, the plurality of corrected images includes a second corrected image that has been subjected to the contrast correction process, the pre-processing further includes a third process of calculating a value of a contrast correction amount included in a contrast correction formula used in the contrast correction process based on the tissue image, the second corrected image, and the contrast correction formula; The inverse transformation process further includes a second inverse transformation process based on the contrast correction formula in which the contrast correction amount is the value. The inspection system of claim 1 .

3. the plurality of corrected images include a third corrected image corrected by both the gamma correction process and the contrast correction process, The pretreatment is a fourth process for obtaining a third inverse-transformed image by performing an inverse transformation process on the third corrected image in the order of the first inverse transformation process and the second inverse transformation process, and a fourth inverse-transformed image by performing an inverse transformation process on the third corrected image in the order of the second inverse transformation process and the first inverse transformation process; a fifth process of identifying an order of the first inverse conversion process and the second inverse conversion process in the inverse conversion process by comparing a third difference in luminance values ​​between the tissue image and the third inverse conversion image and a fourth difference in luminance values ​​between the tissue image and the fourth inverse conversion image, In the inverse conversion process, the first inverse conversion process and the second inverse conversion process are performed in accordance with the order specified in the fifth process. The inspection system of claim 2 .

4. An inspection device for a target component made of heat-treated carbon steel or alloy steel, comprising: A process of receiving a corrected image obtained by performing a correction process on a tissue image obtained by observing a cross section of the target member under a microscope, and the tissue image; an inverse transformation process for performing an inverse transformation of the correction process on the corrected image; a quality inspection process for inspecting the quality of the target component based on the inverse transformation tissue image obtained by the inverse transformation process; A pre-processing step performed before the inverse transformation process; a processing unit for executing the correction process includes a gamma correction process performed using either a first gamma correction formula or a second gamma correction formula different from the first gamma correction formula, the corrected image includes a first corrected image that has been subjected to the gamma correction process, The pretreatment is a first process for obtaining a first inverse-transformed image obtained by performing an inverse transformation process on the first corrected image based on the first gamma correction formula, and a second inverse-transformed image obtained by performing an inverse transformation process on the first corrected image based on the second gamma correction formula; a second process of identifying whether the correction formula used in the gamma correction process is the first gamma correction formula or the second gamma correction formula by comparing a first difference in luminance values ​​between the tissue image and the first inverse transformed image and a second difference in luminance values ​​between the tissue image and the second inverse transformed image, The inverse conversion process includes a first inverse conversion process based on the specified gamma correction formula. Inspection equipment.

Citation Information

Patent Citations

  • Manufacturing method for learning model used for quality evaluation, learning system, quality evaluation system, and quality evaluation method

    JP2021144402A