Object texture measuring device

Abstract

An object texture measuring device comprises: a visual detection unit for detecting the visual texture of an object; a tactile detection unit for detecting the tactile texture of the object; and an acquisition unit for acquiring the detection results of the visual detection unit and the tactile detection unit respectively while causing the visual detection unit and the tactile detection unit to move relative to the object respectively.

Description

Technical Field

[0001] The invention relates to a device for measuring the texture of an object. Background Art

[0002] Patent document 1 proposes a machine learning assistance device, which comprises: an information generating unit, which generates a plurality of virtual information corresponding to the output information of a plurality of sensors; a learning assistance unit, which uses the plurality of virtual information generated by the information generating unit to enable an artificial intelligence device to learn; and a verification assistance unit, which uses the plurality of virtual information to verify the action of the artificial intelligence device after learning.

[0003] In addition, Patent Document 2 proposes a surface hardness measuring device for measuring hardness information of a real object corresponding to a virtual object in a virtual space that is prompted to a user by tactile force, the surface hardness measuring device comprising: a 3D shape measuring unit for measuring the 3D shape of the real object; a displacement measuring unit for measuring the displacement of each preset position on the surface of the real object based on the 3D shape information obtained by the 3D shape measuring unit; and a measurement control unit for estimating the viscoelastic impedance of each measured position of the real object based on the ratio or displacement amount of the displacement obtained by the displacement measuring unit, and obtaining the hardness information of each specified position of the real object through the estimated viscoelastic impedance.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2017-204276

[0005] Patent Document 2: Japanese Patent Application Publication No. 2013-531022 Summary of the invention

[0006] There are technologies for estimating visual texture from images and technologies for estimating tactile texture from images, but since each texture is detected separately, it is impossible to obtain the correspondence between their positions. The purpose of the present invention is to provide an object texture measurement device that can measure visual texture and tactile texture while obtaining the correspondence between their positions.

[0007] The object texture measuring device described in Scheme 1 includes: a visual detection unit for detecting the visual texture of the object; a tactile detection unit for detecting the tactile texture of the object; and an acquisition unit for acquiring the detection results of the visual detection unit and the tactile detection unit respectively while causing the visual detection unit and the tactile detection unit to move relative to the object respectively.

[0008] The invention described in claim 2 is the invention described in claim 1, further comprising a moving unit that moves the visual detection unit and the tactile detection unit relative to the object.

[0009] The invention described in Scheme 3, in the invention described in Scheme 1 or 2, further includes: a detection unit, which detects the relative movement distance of the visual detection unit and the tactile detection unit relative to the object; and a processing unit, which uses the detection result of the detection unit to align the respective detection positions of the detection result of the visual detection unit and the detection result of the tactile detection unit, and sets the direction of the relative movement to a predetermined direction.

[0010] The invention described in Scheme 4, in the invention described in Scheme 1 or 2, further includes: a detection unit, which detects the relative movement distance and the respective movement direction of the visual detection unit and the tactile detection unit relative to the object; and a processing unit, which uses the detection result of the detection unit to align the respective detection positions of the detection result of the visual detection unit and the detection result of the tactile detection unit.

[0011] The invention described in claim 5 is the invention described in any one of claims 1 to 4, wherein the tactile detection unit includes a plurality of sensors that detect a plurality of types of surface information indicating a tactile texture of the surface of the object.

[0012] The invention described in Option 6 is the invention described in Option 5, wherein the plurality of sensors include a sensor that contacts the object to detect surface information representing the tactile texture of the surface of the object; and a sensor that detects surface information representing the tactile texture of the surface of the object without contacting the object.

[0013] The invention described in claim 7 is the invention described in any one of claims 1 to 6, wherein the visual detection unit detects the visual texture by detecting specular reflection light and diffuse reflection light from the object.

[0014] The invention described in Scheme 8 In the invention described in Scheme 7, the visual detection unit includes: a single light source that irradiates light to the object; a first camera unit that is set at a position that receives mirror-reflected light irradiated from the light source and reflected from the object, and photographs the surface of the object; and a second camera unit that is set at a position that receives diffusely reflected light irradiated from the light source and reflected from the object, and photographs the surface of the object.

[0015] The invention described in Scheme 9 In the invention described in Scheme 7, the visual detection unit includes: a single camera unit, which is set at a position to receive reflected light from an object and photographs the surface of the object; a first light source, which is set at a position relative to the camera unit where specular reflected light from the object is incident and irradiates light to the object; and a second light source, which is set at a position relative to the camera unit where diffuse reflected light from the object is incident and irradiates light to the object.

[0016] The object texture measuring device described in Option 10 includes a processor, which performs the following processing: a first process for detecting the visual texture of the object, a second process for detecting the tactile texture of the object, and a third process for aligning the detection positions of the detection results of the visual texture detected by the first process and the detection results of the tactile texture detected by the second process.

[0017] Effects of the Invention

[0018] According to the first aspect of the present invention, it is possible to provide an object texture measurement device capable of measuring visual texture and tactile texture while obtaining correspondence between positions.

[0019] According to the second aspect of the present invention, it is possible to save the time and effort of moving the visual detection unit and the tactile detection unit relative to the object.

[0020] According to the third aspect of the present invention, the correspondence between the detection positions of the visual detection unit and the tactile detection unit can be acquired with higher accuracy than when only the detection results of each of the visual detection unit and the tactile detection unit are acquired.

[0021] According to the fourth aspect of the present invention, the correspondence between the detection positions of the visual detection unit and the tactile detection unit can be acquired with higher accuracy than when only the detection results of each of the visual detection unit and the tactile detection unit are acquired.

[0022] According to the fifth aspect of the present invention, it is possible to detect a more detailed tactile texture than when only one type of surface information is detected.

[0023] According to the sixth aspect of the present invention, two types of tactile textures at the same position can be detected simultaneously.

[0024] According to the seventh aspect of the present invention, the reflection characteristics of an object can be detected as a visual texture.

[0025] According to the eighth aspect of the present invention, it is possible to simultaneously capture specular reflection and diffuse reflection.

[0026] According to the ninth aspect of the present invention, it is possible to capture both specular reflection and diffuse reflection using a single imaging unit.

[0027] According to the tenth aspect of the present invention, it is possible to provide an object texture measurement device capable of measuring visual texture and tactile texture while obtaining correspondence between positions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0029] Figure 1is a diagram showing a schematic structure of an object texture measuring device according to the present embodiment;

[0030] Figure 2 is a block diagram showing the structure of a control system of the object texture measuring device according to the present embodiment;

[0031] Figure 3 is a flowchart showing an example of the flow of processing performed in the control unit of the object texture measurement device according to the present embodiment;

[0032] Figure 4 is a diagram showing an example of using the measurement results of the object texture measurement device according to the present embodiment;

[0033] Figure 5 It is a diagram showing a schematic configuration of a modified example of the object texture measurement device according to the present embodiment.

[0034] Explanation of symbols

[0035] 10, 11-object texture measuring device, 12, 13-light source, 14, 16-image sensor, 18-force sensor, 20-object, 22-moving unit, 24-moving distance detection unit, 26-control unit, 28-measurement position correction unit, 30-corresponding association establishment processing unit, 32-storage unit. DETAILED DESCRIPTION

[0036] Hereinafter, an example of the present embodiment will be described in detail with reference to the drawings. Figure 1 1 is a diagram showing a schematic configuration of an object texture measurement device according to the present embodiment.

[0037] The object texture measurement device 10 according to the present embodiment includes a light source 12, image sensors 14 and 16, and a force sensor 18. The object texture measurement device 10 measures the texture of the object while moving relative to an object 20 to be measured. Figure 1 The object 20 is measured while moving in the direction of arrow X. In addition, the light source 12 and the image sensors 14 and 16 correspond to the visual detection unit, the image sensor 14 corresponds to the second imaging unit, and the image sensor 16 corresponds to the first imaging unit. Furthermore, the force sensor 18 corresponds to the tactile detection unit.

[0038] The object texture measuring device 10 optically reads the surface characteristics of the object 20 and generates image information indicating the reading result, and measures the surface state of the object 20 and generates surface information indicating the measurement result. The image information generated by the object texture measuring device 10 includes diffuse reflection image information based on diffuse reflection light and specular reflection image information based on specular reflection light.

[0039] In the present embodiment, the object 20 is described as a planar object, but is not limited to a planar object, and may be a three-dimensional object.

[0040] The object texture measuring device 10 is configured with the components shown in the figure in a predetermined width in a direction perpendicular to the paper surface. This direction is the main scanning direction of the object texture measuring device 10, and the direction indicated by the arrow X in the figure is the width scanning direction of the object texture measuring device 10.

[0041] The light source 12 irradiates the object 20 with light for measuring the visual texture of the surface of the object 20. The light source 12 is set at a position where the light is irradiated at an incident angle of 45 degrees relative to the measurement surface of the object 20. As an example of the light source 12, white light such as a fluorescent lamp or a rare gas fluorescent lamp (xenon fluorescent lamp, etc.) is suitable. In addition, the light source 12 can arrange a plurality of white LEDs along the main scanning direction, and use a diffusion plate, etc. to make the brightness distribution in the main scanning direction uniform.

[0042] The image sensors 14 and 16 are suitable for, for example, line sensors or surface sensors, and detect the reflection characteristics (for example, BRDF: Bidirectional Reflectance Distribution) of the object 20 as the visual texture of the object 20. Light irradiated from the light source 12 and reflected by the object 20 is incident on the image sensors 14 and 16, and an image signal corresponding to the incident light is generated. In the present embodiment, the image sensor 14 is arranged in the normal direction of the object 20, and diffusely reflected light of the light irradiated from the light source 12 to the object 20 is incident. On the other hand, the image sensor 16 is arranged in a direction of 45 degrees relative to the normal direction of the object 20, and specularly reflected light of the light irradiated from the light source to the object 20 is incident. The image sensors 14 and 16 are composed of, for example, photoelectric conversion elements such as CCD (Charge Coupled Device) linear image sensors or CMOS (Complementary Metal Oxide Semiconductor) image sensors, and convert the received light into a signal indicating its intensity. The image sensors 14 and 16 include color filters and generate image information indicating the color of the object 20. The image sensor 14 outputs diffuse reflection image information obtained by receiving diffuse reflection light, while the image sensor 16 outputs specular reflection image information obtained by receiving specular reflection light.

[0043] The force sensor 18 moves relative to the object 20 while in contact with the object 20, thereby detecting surface information such as the smoothness, unevenness, hardness, etc. of the surface of the object 20 as a tactile texture. A plurality of force sensors 18 are arranged along the main scanning direction. In addition, as an example of a tactile detection unit, a force sensor 18 that contacts the object 20 for detection is described, but it is not limited to this. For example, an ultrasonic sensor, a surface resistance sensor, etc. that detects without contacting the object 20 can also be used. In addition, any sensor can be used as long as it can be associated with and calibrated with the Young's modulus measuring device that serves as a reference device. Alternatively, these multiple sensors can also be set in combination. For example, a sensor that contacts the object 20 for detection and a sensor that detects without contacting the object 20 can be combined to simultaneously detect multiple types of surface information at the same position.

[0044] Next, the configuration of a control system of the object texture measurement device 10 according to the present embodiment will be described. Figure 2 1 is a block diagram showing a configuration of a control system of the object texture measurement device 10 according to the present embodiment.

[0045] The object texture measurement device 10 includes a light source 12 , image sensors 14 and 16 , a force sensor 18 , a moving unit 22 , a moving distance detection unit 24 as a detection unit, a control unit 26 , and a storage unit 32 , and each of the components is electrically connected to a system bus 34 .

[0046] The moving unit 22 moves the object texture measurement device 10 relative to the object 20 when measuring the object 20. For example, the moving unit 22 includes an actuator such as a motor, and moves the object texture measurement device 10 relative to the object 20 by driving the actuator.

[0047] The moving distance detection unit 24 detects the moving distance of the object texture measurement device 10 relative to the object 20 . For example, the moving distance may be detected by a rotary encoder or the like, or may be detected based on image information obtained by the image sensor 14 .

[0048] The control unit 26 is composed of a computer including a CPU (Central Processing Unit) as an example of a processor, a ROM (Read Only Memory) and a RAM (Random Access Memory). The CPU is in charge of the overall operation of the object texture measuring device 10. The RAM is used as a work area when executing various programs based on the CPU. The ROM stores various control programs or various parameters in advance. In this embodiment, the CPU executes the program stored in the ROM, thereby executing the functions of the measurement position correction unit 28 and the corresponding association establishment processing unit 30. In addition, the measurement position correction unit 28 and the corresponding association establishment processing unit 30 correspond to the processing unit.

[0049] Since the detection positions of the image sensors 14 and 16 and the detection position of the force sensor 18 are physically different relative to the object 20, the measurement position correction unit 28 performs correction to align the measurement positions based on the detection result of the moving distance detection unit 24 in order to align the correspondence of the respective positions. In this embodiment, after the object texture measurement device 10 moves a preset distance, the position measured by the force sensor 18 is measured by the image sensors 14 and 16. And the detection results of the image sensors 14 and 16 when the force sensor 18 performs the measurement until the movement of the preset distance is detected are corrected as the measurement results of the same position of the object 20, thereby performing correction to align the measurement positions of the image sensors 14 and 16 with the measurement position of the force sensor 18. In this way, the visual texture and the tactile texture are measured at the same time.

[0050] The association processing unit 30 associates the image information of the measurement results of the image sensors 14 and 16 that measured the same position of the object 20 with the surface information of the measurement result of the load cell 18 based on the calibration result of the measurement position calibration unit 28 .

[0051] The storage unit 32 stores the image information and the surface information associated with each other by the association processing unit 30. The storage unit 32 may be, for example, a hard disk drive (HDD) or a non-volatile memory such as a flash memory.

[0052] Next, specific processing performed by the control unit 26 of the object texture measurement device 10 according to the present embodiment will be described. Figure 3 : is a flowchart showing an example of the process flow performed by the control unit 26 of the object texture measurement device 10 according to the present embodiment. Figure 3 The processing starts when, for example, a start button (not shown) is operated to instruct the start of measurement of the object 20.

[0053] In step S100, the control unit 26 turns on the light source 12 and drives the moving unit 22 to start measurement, and then the process proceeds to step S102. Thus, the object texture measurement device 10 moves relative to the object 20 to start measuring visual texture and tactile texture.

[0054] In step S102, the control unit 26 acquires the detection result of the load cell 18 and transitions to step S102. That is, the surface information is acquired from the load cell 18. The process of step S102 corresponds to the second process.

[0055] In step S104, the control unit 26 acquires the detection results of the image sensors 14 and 16 and transitions to step S106. That is, diffuse reflection image information is acquired from the image sensor 14, and specular reflection image information is acquired from the image sensor 16. In addition, the processing of step S104 corresponds to the first processing. In addition, the processing of steps S102 and S104 corresponds to the acquisition unit.

[0056] In step S106, the control unit 26 establishes a correspondence between the detection results of the image sensors 14 and 16 and the detection results of the force sensor 18 and transitions to step S108. Specifically, the detection positions of the image sensors 14 and 16 and the detection positions of the force sensor 18 are physically different relative to the object 20, so in order to align the respective detection positions, correction is performed to align the detection positions based on the detection results of the moving distance detection unit 24. And, the correspondence establishment processing unit 30 establishes a correspondence between the image information of the detection results of the image sensors 14 and 16 that detected the same position of the object and the surface information of the detection results of the force sensor 18 based on the correction results of the measurement position correction unit 28. In addition, the processing of step S106 corresponds to the third processing.

[0057] In step S108 , the control unit 26 stores the association result in the storage unit 32 and the process proceeds to step S110 .

[0058] In step S110, the control unit 26 determines whether the measurement is to be terminated. This determination is, for example, whether a stop button (not shown) is operated to instruct the termination of the measurement of the object 20. If the determination is negative, the process returns to step S102 and the above-mentioned process is repeated, and if the determination is positive, the series of processes is terminated.

[0059] By performing such processing, diffuse reflection image information and specular reflection image information detected by the image sensors 14 and 16 are obtained as the measurement results of visual texture, and surface information detected by the force sensor 18 is obtained as the measurement results of tactile texture. In addition, correction is performed to align the measurement positions of the visual texture measurement results and the tactile texture measurement results to establish a corresponding association. That is, the visual texture and the tactile texture are measured while obtaining the corresponding positions.

[0060] The measurement result of the object texture measuring device 10 is, for example, Figure 4 As shown, by inputting to the machine learning unit 36 ​​and performing deep learning on the visual texture and the tactile texture at the same time, it is applied to a service that reproduces the visual texture and the tactile texture in a device that reproduces the visual texture and the tactile texture based on the captured image.

[0061] In the above embodiment, an example in which the object texture measurement apparatus 10 is moved relative to the object 20 is described, but the present invention is not limited thereto, and the object 20 may be moved relative to the object texture measurement apparatus 10 .

[0062] Moreover, in the above-described embodiment, the structure is configured to include the moving unit 22 and relatively move the object texture measuring device 10 with respect to the object 20, but it is not limited thereto. For example, the moving unit 22 may be omitted, and the object texture measuring device 10 or the object 20 may be manually moved.

[0063] Moreover, in the above-described embodiment, an example in which the moving direction of the object texture measuring device 10 relative to the object has been determined is given for explanation, but it is not limited thereto. For example, when the moving direction is arbitrary or manual movement is involved, in order to make the measurement results of visual texture correspond to the measurement results of tactile texture, it is necessary to detect the moving direction. And when the moving direction is arbitrary or manual movement is involved, for example, the moving distance detection unit 24 also detects the moving direction, and the measurement position correction unit 28 corrects the measurement positions according to the moving distance and the moving direction detected by the moving distance detection unit 24.

[0064] Moreover, in the above-described embodiment, the light source 12 is disposed at a position where light is irradiated at an incident angle of 45 degrees with respect to the normal direction of the object 20, the image sensor 14 is disposed in the normal direction of the object 20, and the image sensor 16 is disposed in a direction 45 degrees with respect to the normal direction of the object 20, but it is not limited thereto. As long as the image sensor 14 receives the diffuse reflected light and the image sensor 16 receives the specular reflected light, the light source 12 and the image sensors 14 and 16 can be arranged at any positions.

[0065] Moreover, in the above-described embodiment, an example in which the diffuse reflection image information and the specular reflection image information are obtained as the measurement results of visual texture by the light source 12, the image sensor 14, and the image sensor 16 is given for explanation, but the structure is not limited thereto. For example, as Figure 5 shown in the object texture measuring device 11, it may be configured to include the light source 12 as the first light source, the light source 13 as the second light source, and the image sensor 16 as the imaging unit. At this time, the light source 12 and the image sensor 16 are arranged in the same manner as in the above-described embodiment, and the light source 13 is arranged in the normal direction of the object 20. And by performing two readings, the diffuse reflection image information and the specular reflection image information are obtained. That is, only the light source 12 is lit to obtain the detection result of the image sensor 16, thereby obtaining the specular reflection image information. On the other hand, only the light source 13 is lit to obtain the detection result of the image sensor 16, thereby obtaining the diffuse reflection image information.

[0066] Moreover, in the above-described embodiment, the measurement position correction unit 28 is provided, but depending on the measurement resolution or the measurement positions of visual texture and tactile texture, etc., the structure may be configured to omit the moving distance detection unit 24 and the measurement position correction unit 28.

[0067] Furthermore, in each of the above-mentioned embodiments, the processor refers to a processor in a broad sense, including general-purpose processors (such as CPU: Central Processing Unit, etc.) and special-purpose processors (such as GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, editable logic devices, etc.).

[0068] Furthermore, the actions of the processors in the above embodiments are not performed by only one processor, but can also be performed by cooperation of multiple processors located in physically separated positions. Furthermore, the order of the actions of the processors is not limited to the order described in the above embodiments, and can be changed appropriately.

[0069] Furthermore, the processing performed by the control unit 26 in the above-mentioned embodiment may be performed by software, by hardware, or by a combination of the two. Furthermore, the processing performed by the control unit 26 may be stored in a storage medium as a program and circulated.

[0070] Furthermore, the present invention is not limited to the above, and various modifications other than the above are of course possible without departing from the gist of the present invention.

[0071] The above-mentioned embodiments of the present invention are provided for the purpose of illustration and description. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed methods. Obviously, various modifications and changes are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily understand the principles of the present invention and its application. Thus, other technicians in the art can understand the present invention through various variations optimized for specific uses assumed to be various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

Claims

1. A device for measuring texture of an object, include: The visual inspection unit detects the visual texture of objects; A tactile detection unit, detecting the tactile texture of the object; an acquisition unit, which acquires detection results of the visual detection unit and the tactile detection unit respectively while causing the visual detection unit and the tactile detection unit to move relative to the object respectively; A detection unit configured to detect a relative movement distance of each of the visual detection unit and the tactile detection unit relative to the object; as well as The processing unit performs a process of aligning detection positions of the detection result of the visual detection unit and the detection result of the tactile detection unit using the detection result of the detection unit.

2. The object texture measuring device according to claim 1, further comprising: include: The moving unit moves the visual detection unit and the tactile detection unit relative to the object.

3. The object texture measuring device according to claim 1 or 2, in, The direction of the relative movement is set to a preset direction.

4. The object texture measuring device according to claim 1 or 2, in, The detection unit detects a movement direction of each of the visual detection unit and the tactile detection unit.

5. The object texture measuring device according to claim 1 or 2, in, The tactile detection unit includes a plurality of sensors that detect a plurality of types of surface information indicating tactile textures of the surface of the object.

6. The object texture measuring device according to claim 5, in, The plurality of sensors include a sensor that contacts the object to detect surface information representing a tactile texture of the surface of the object; and a sensor that detects surface information representing a tactile texture of the surface of the object without contacting the object.

7. The object texture measuring device according to claim 1 or 2, in, The visual detection unit detects the visual texture by detecting specular reflection light and diffuse reflection light from the object respectively.

8. The object texture measuring device according to claim 7, in, The visual detection unit includes: a single light source that irradiates light toward the object; a first camera unit that is disposed at a position that receives specularly reflected light irradiated from the light source and reflected from the object, and photographs the surface of the object; and a second camera unit that is disposed at a position that receives diffusely reflected light irradiated from the light source and reflected from the object, and photographs the surface of the object.

9. The object texture measuring device according to claim 7, in, The visual detection unit includes: a single camera unit, which is arranged at a position to receive reflected light from an object and photograph the surface of the object; a first light source, which is arranged at a position relative to the camera unit where specular reflected light from the object is incident and irradiates light to the object; and a second light source, which is arranged at a position relative to the camera unit where diffuse reflected light from the object is incident and irradiates light to the object.

10. An object texture measurement device, comprising a processor, wherein the processor performs the following processing: A first process for detecting a visual texture of an object, a second process for detecting a tactile texture of the object, and a third process for aligning detection positions of the detection result of the visual texture detected by the first process and the detection result of the tactile texture detected by the second process.

Images