Calibration method, device, electronic device, medium and program product

Abstract

The present invention provides a calibration method, device, electronic device, computer-readable storage medium and computer program product for a curved contact sensor. It can be applied to the field of robot and sensor technology. The calibration method includes using a camera intrinsic parameter and distortion coefficient calibration method to obtain the intrinsic parameter and distortion coefficient of a monocular camera; using a camera extrinsic parameter calibration method to obtain the extrinsic parameter of the monocular camera relative to a contact module; obtaining a predetermined m sample data; inputting the m sample data into a pre-constructed neural network model, training to obtain the model parameters of the neural network model, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multilayer perceptron; and using the model parameters of the neural network model as the calibration parameters of the curved contact sensor. The calibration method can calibrate the parameters of the proposed curved contact sensor.

Description

Technical Field

[0001] The present invention relates to the field of sensor technology, and more specifically, to a calibration method, device, electronic equipment, computer-readable storage medium and computer program product for a curved contact sensor. Background Art

[0002] In the prior art, tactile sensors are essential for robot operation because they can provide accurate feedback about contact position and surface features of objects, etc., so that the robot can perform more delicate and accurate operations. The present invention proposes a calibration method for a curved surface contact sensor for the proposed curved surface contact sensor, aiming to calibrate the curved surface contact sensor so that the curved surface contact sensor can accurately achieve the detection effect.

[0003] The curved surface contact sensor proposed by the present invention comprises a housing, a monocular camera, a contact module and a lighting module. The housing has an installation cavity, and the installation cavity has an open mouth; the monocular camera is arranged in the installation cavity; the contact module comprises a black coating and a translucent silicone layer, the black coating is suitable for contacting with the object to be measured, the contact surface of the black coating and the object to be measured and the surface opposite to the contact surface are both formed into a curved surface, the black coating is coated on the translucent silicone layer, the translucent silicone layer is connected to the housing, blocks the open mouth, and faces the monocular camera; the lighting module is arranged in the housing, and is used to illuminate the contact module. Summary of the invention

[0004] In view of this, the present invention provides a calibration method, device, electronic device, computer-readable storage medium and computer program product capable of calibrating parameters of a curved surface contact sensor.

[0005] One aspect of the present invention provides a calibration method for a curved contact sensor, wherein the curved contact sensor includes a monocular camera and a contact module, and the calibration method includes: using a camera intrinsic parameter and distortion coefficient calibration method to obtain the intrinsic parameter and distortion coefficient of the monocular camera; using a camera extrinsic parameter calibration method to obtain the extrinsic parameter of the monocular camera relative to the contact module; obtaining m predetermined sample data, wherein each sample data includes a grayscale image of the contact deformation of the contact module captured by the monocular camera after calibration of the intrinsic parameter, the distortion coefficient and the extrinsic parameter, and m is an integer greater than or equal to 1; inputting the m sample data into a pre-constructed neural network model, and training to obtain model parameters of the neural network model, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multilayer perceptron; and using the model parameters of the neural network model as the calibration parameters of the curved contact sensor.

[0006] In some examples, the step of predetermining m sample data includes: splicing and buckling n pre-manufactured first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different protrusion arrangement, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact area of ​​the first sample mold and the second sample mold with the contact module is equal, and n is an integer greater than or equal to 1; using the monocular camera calibrated with the intrinsic parameters, the distortion coefficient and the extrinsic parameters to capture the preprocessed grayscale image of the second sample mold spliced ​​and buckled with each of the first sample molds on the contact module and the contact deformation of the contact module; removing half of the image captured by the contact module and the second sample mold in each of the preprocessed grayscale images, retaining the other half of the image, to obtain a cropped image; splicing every two cropped images of the n cropped images to obtain m sample data, wherein m=n n .

[0007] In some examples, the step of predetermining m sample data includes: splicing and buckling n pre-manufactured first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different protrusion arrangement, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact area of ​​the first sample mold and the second sample mold with the contact module is equal, and n is an integer greater than or equal to 1; using the monocular camera calibrated with the intrinsic parameters, the distortion coefficient and the extrinsic parameters to capture the second sample mold and each of the first sample molds spliced ​​and buckled on the contact module, and the preprocessed grayscale image of the contact module contact deformation; removing half of the image captured by the contact module and the second sample mold in each of the preprocessed grayscale images, retaining the other half of the image, to obtain a cropped image; splicing every two cropped images of the n cropped images to obtain m spliced ​​images, wherein m=n n ; Subtract the reference image from the m stitched images to obtain m sample data, wherein the reference image is an image captured when the contact module is in a non-contact state.

[0008] In some examples, the first layer of the two-layer convolutional neural network includes 64 channels, each of which has a kernel size of 7, and the second layer of the convolutional neural network includes 128 channels, each of which has a kernel size of 7. The first layer of the two-layer multilayer perceptron reduces the feature dimension of each pixel of the sample data to 64 dimensions, and the second layer of the multilayer perceptron reduces the 64 dimensions to 1 dimension.

[0009] In some examples, the step of obtaining the external parameters of the monocular camera relative to the contact module using a camera external parameter calibration method includes: the monocular camera captures a model grayscale image of a 1:1 model after the insert is inserted k times, wherein the 1:1 model is pre-manufactured, and the 1:1 model replaces the contact module in the curved contact sensor, and the outer surface of the 1:1 model is provided with g insertion holes with known world coordinate positions, which are suitable for inserting the insert, and the layout of the insert inserted each time on the 1:1 model is different, k is an integer greater than or equal to 1, and g is an integer greater than or equal to 1; the external parameters of the monocular camera are calculated according to the world coordinate position of the insert in each model grayscale image, the intrinsic parameters of the monocular camera, and the distortion coefficient.

[0010] In some examples, the camera intrinsic parameter and distortion coefficient calibration method includes a checkerboard calibration method.

[0011] Another aspect of the present invention provides a calibration device for a curved contact sensor, the curved contact sensor comprising a monocular camera and a contact module, the calibration device comprising: a first determination module, the first determination module being used to execute a calibration method using camera intrinsic parameters and distortion coefficients to obtain the intrinsic parameters and distortion coefficients of the monocular camera; a second determination module, the second determination module being used to execute a calibration method using camera extrinsic parameters to obtain the extrinsic parameters of the monocular camera relative to the contact module; an acquisition module, the acquisition module being used to execute acquisition of predetermined m sample data, wherein each of the sample data comprises a grayscale image of the contact deformation of the contact module captured by the monocular camera after calibration of the intrinsic parameters, the distortion coefficients and the extrinsic parameters, and m is an integer greater than or equal to 1; a training module, the training module being used to execute inputting the m sample data into a pre-constructed neural network model to train and obtain model parameters of the neural network model, wherein the neural network model comprises a two-layer convolutional neural network and a two-layer multilayer perceptron; a third determination module, the third determination module being used to execute using the model parameters of the neural network model as calibration parameters of the curved contact sensor.

[0012] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and one or more memories for storing executable instructions, wherein when the executable instructions are executed by the processors, the method described above is implemented.

[0013] Another aspect of the present invention provides a computer-readable storage medium, characterized in that executable instructions are stored on the storage medium, and when the instructions are executed by a processor, the method described above is implemented.

[0014] Another aspect of the present invention provides a computer program product, characterized in that it comprises a computer program, wherein the computer program comprises one or more executable instructions, and when the executable instructions are executed by a processor, the method described above is implemented.

[0015] In some examples, by utilizing the camera intrinsic parameter and distortion coefficient calibration method, the intrinsic parameters and distortion coefficient of the monocular camera can be obtained; by utilizing the camera extrinsic parameter calibration method, the extrinsic parameters of the monocular camera relative to the contact module can be obtained; by obtaining predetermined m sample data, and inputting the m sample data into a pre-built neural network model, the model parameters of the neural network model can be trained, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multilayer perceptron; the model parameters of the neural network model can be used as calibration parameters of the curved contact sensor, thereby, the curved contact sensor proposed in the present invention can work normally, so that the curved contact sensor can accurately achieve the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0017] Figure 1 A flow chart schematically shows a calibration method for a curved surface contact sensor according to an embodiment of the present invention;

[0018] Figure 2 A flowchart schematically shows the steps of obtaining the external parameters of a monocular camera relative to a contact module by using a camera external parameter calibration method according to an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of the structure of a 1:1 model according to an embodiment of the present invention is shown;

[0020] Figure 4 A schematic diagram of the structure after the insert according to an embodiment of the present invention is inserted into a 1:1 model is shown;

[0021] Figure 5 A flowchart schematically shows the steps of predetermining m sample data according to an embodiment of the present invention;

[0022] Figure 6 A schematic diagram of the structure of a curved surface contact sensor according to an embodiment of the present invention is shown;

[0023] Figure 7 Schematically shows a structural diagram of a first sample mold according to an embodiment of the present invention;

[0024] Figure 8 Schematically shows a structural diagram of a second sample mold according to an embodiment of the present invention;

[0025] Fig. 9 A schematic diagram of a structure in which a first sample mold and a second sample mold are spliced ​​and buckled on a contact module according to an embodiment of the present invention is shown;

[0026] Fig.10 A flowchart schematically shows the steps of predetermining m sample data according to an embodiment of the present invention;

[0027] Fig.11 The structure block diagram of the calibration device of the curved contact sensor according to the embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0028] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0029] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and they do not violate public order and good morals. In the technical solution of the present invention, the acquisition, collection, storage, use, processing, transmission, provision, disclosure and application of data are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and they do not violate public order and good morals.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] When using expressions such as "at least one of A, B or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the said features.

[0032] In the prior art, tactile sensors are essential for robot operation because they can provide accurate feedback about contact position and surface features of objects, etc., so that the robot can perform more delicate and accurate operations. The present invention proposes a calibration method for a curved surface contact sensor for the proposed curved surface contact sensor, aiming to calibrate the curved surface contact sensor so that the curved surface contact sensor can accurately achieve the detection effect.

[0033] like Figure 1-Figure 11 As shown, the embodiments of the present invention provide a calibration method, device, electronic device, computer-readable storage medium and computer program product for a curved contact sensor 100 .

[0034] The curved surface contact sensor 100 proposed by the present invention comprises a housing, a monocular camera, a contact module 101 and a lighting module. The housing has an installation cavity, and the installation cavity has an open mouth; the monocular camera is arranged in the installation cavity; the contact module comprises a black coating and a translucent silicone layer, the black coating is suitable for contacting with the object to be measured, the contact surface of the black coating and the object to be measured and the surface opposite to the contact surface are both formed as curved surfaces, the black coating is coated on the translucent silicone layer, the translucent silicone layer is connected to the housing, blocks the open mouth, and faces the monocular camera; the lighting module is arranged in the housing, and is used to illuminate the contact module.

[0035] like Figure 1 As shown, the calibration method of the curved contact sensor 100 according to the embodiment of the present invention includes S110 to S150.

[0036] In S110, the camera intrinsic parameter and distortion coefficient calibration method is used to obtain the intrinsic parameter and distortion coefficient of the monocular camera.

[0037] As a feasible method, the camera intrinsic parameter and distortion coefficient calibration method includes a checkerboard calibration method. The checkerboard calibration method can be used to easily obtain the intrinsic parameters and distortion coefficients of the monocular camera.

[0038] In S120 , the camera extrinsic parameter calibration method is used to obtain the extrinsic parameters of the monocular camera relative to the contact module 101 .

[0039] As a feasible approach, Figure 2 As shown, S120 uses a camera extrinsic parameter calibration method to obtain the extrinsic parameters of the monocular camera relative to the contact module 101, including S121 and S122.

[0040] In S121, the monocular camera captures a grayscale image of the 1:1 model 200 after the insert 300 is inserted k times, wherein the 1:1 model 200 is pre-manufactured, and the 1:1 model 200 replaces the contact module 101 in the curved contact sensor 100. The outer surface of the 1:1 model 200 is provided with g insertion holes 201 with known world coordinate positions, which are suitable for inserting the insert 300. The layout of the insert 300 inserted each time on the 1:1 model 200 is different, k is an integer greater than or equal to 1, and g is an integer greater than or equal to 1. Figure 3 Schematic diagram of the structure of a 1:1 model 200 according to an embodiment of the present invention. Figure 4 It is a schematic structural diagram of the insert 300 of the embodiment of the present invention after being inserted into the 1:1 model 200.

[0041] The layout of the insert 300 on the 1:1 model 200 can be understood as a combination of the number and position of the insert 300 inserted into the 1:1 model 200. For example, the world coordinate positions of the g insertion holes 201 of the 1:1 model 200 are position 1, position 2, position 3, position 4, and position 5 respectively; the layout of the insert 300 inserted for the first time on the 1:1 model 200 is: the insert 300 is inserted into position 1, position 3, and position 4; the layout of the insert 300 inserted for the second time on the 1:1 model 200 is: the insert 300 is inserted into position 2, position 3, and position 4; the layout of the insert 300 inserted for the third time on the 1:1 model 200 is: the insert 300 is inserted into position 1, position 2, and position 3, and so on. The above three layouts of the insert 300 on the 1:1 model 200 are merely examples for facilitating the understanding of the layout of the insert 300 on the 1:1 model 200, and cannot be understood as a limitation to the present invention.

[0042] In S122 , the external parameters of the monocular camera are calculated according to the world coordinate position of the insert 300 in each model grayscale image, the internal parameters of the monocular camera, and the distortion coefficient.

[0043] S121 and S122 can facilitate the step of obtaining the external parameters of the monocular camera relative to the contact module 101 by using the camera external parameter calibration method.

[0044] In S130, predetermined m sample data are obtained, wherein each sample data includes a grayscale image of the contact deformation of the contact module 101 captured by a monocular camera after calibration of internal parameters, distortion coefficients, and external parameters, and m is an integer greater than or equal to 1.

[0045] As an implementable approach, Figure 5 As shown, the step of predetermining m sample data includes S210~S240.

[0046] In S210, n pre-manufactured first sample molds 400 and second sample molds 500 are spliced ​​and fastened on the contact module 101, wherein the wall surface of each first sample mold 400 in the n first sample molds 400 that contacts the contact module 101 has a different arrangement of protrusions 401, the wall surface of the second sample mold 500 that contacts the contact module 101 is a smooth curved surface, the contact areas of the first sample mold 400 and the second sample mold 500 that contact the contact module 101 are equal, and n is an integer greater than or equal to 1.

[0047] In S220 , a monocular camera with calibrated internal parameters, distortion coefficients and external parameters is used to capture pre-processed grayscale images of the second sample mold 500 and each first sample mold 400 spliced ​​and buckled on the contact module 101 and the contact module 101 being deformed in contact.

[0048] In S230 , half of the image captured when the contact module 101 of each pre-processed grayscale image is in contact with the second sample mold 500 is removed, and the other half of the image is retained to obtain a cropped image.

[0049] In S240, every two cropped images of the n cropped images are spliced ​​to obtain m sample data, where m=n n .

[0050] It should be noted that Figure 6 Schematic diagram of the structure of the curved surface contact sensor 100 according to an embodiment of the present invention. Figure 7 4 is a schematic structural diagram of a first sample mold 400 according to an embodiment of the present invention. Figure 8 is a schematic structural diagram of a second sample mold 500 according to an embodiment of the present invention. Fig. 9 It is a structural schematic diagram of a first sample mold 400 and a second sample mold 500 spliced ​​and buckled on a contact module 101 according to an embodiment of the present invention.

[0051] It can be understood that the portion of the pre-processed grayscale image corresponding to the contact between the first sample mold 400 and the contact module 101 is a shaded portion, and the shadow is the arrangement of the protrusions 401 of the first sample mold 400; since the wall surface of the second sample mold 500 in contact with the contact module 101 is a smooth curved surface, the portion of the pre-processed grayscale image corresponding to the contact between the second sample mold 500 and the contact module 101 is a non-shaded portion. Through S210 to S240, more sample data can be obtained with fewer molds, that is, n+1 molds can be used to obtain n n Sample data, saving time and effort.

[0052] As another possible implementation method, Fig.10 As shown, the step of predetermining m sample data includes S310~S350.

[0053] In S310, n pre-manufactured first sample molds 400 and second sample molds 500 are spliced ​​and fastened on the contact module 101, wherein the wall surface of each first sample mold 400 in the n first sample molds 400 that contacts the contact module 101 has a different arrangement of protrusions 401, the wall surface of the second sample mold 500 that contacts the contact module 101 is a smooth curved surface, the contact areas of the first sample mold 400 and the second sample mold 500 that contact the contact module 101 are equal, and n is an integer greater than or equal to 1.

[0054] In S320 , a monocular camera with calibrated internal parameters, distortion coefficients and external parameters is used to capture pre-processed grayscale images of the second sample mold 500 and each first sample mold 400 spliced ​​and buckled on the contact module 101 and the contact module 101 being deformed in contact.

[0055] In S330 , half of the image captured when the contact module 101 of each pre-processed grayscale image is in contact with the second sample mold 500 is removed, and the other half of the image is retained to obtain a cropped image.

[0056] In S340, every two cropped images of the n cropped images are spliced ​​together to obtain m spliced ​​images, where m=n n .

[0057] In S350, the reference image is subtracted from the m stitched images to obtain m sample data, wherein the reference image is an image captured when the contact module 101 is in a non-contact state.

[0058] It should be noted that Figure 6 Schematic diagram of the structure of the curved surface contact sensor 100 according to an embodiment of the present invention. Figure 7 4 is a schematic structural diagram of a first sample mold 400 according to an embodiment of the present invention. Figure 8 is a schematic structural diagram of a second sample mold 500 according to an embodiment of the present invention. Fig. 9 It is a structural schematic diagram of a first sample mold 400 and a second sample mold 500 spliced ​​and buckled on a contact module 101 according to an embodiment of the present invention.

[0059] It can be understood that the part of the pre-processed grayscale image corresponding to the contact between the first sample mold 400 and the contact module 101 is the shaded part, and the shadow is the arrangement of the protrusions 401 of the first sample mold 400; since the wall surface of the second sample mold 500 in contact with the contact module 101 is a smooth curved surface, the part of the pre-processed grayscale image corresponding to the contact between the second sample mold 500 and the contact module 101 is the non-shaded part. Through S310~S350, more sample data can be obtained with fewer molds, that is, n+1 molds can be used to obtain n nIn addition, the m sample data of the present invention are obtained by subtracting the reference image from the m spliced ​​images, which makes the sample data of the present invention simple and easy to calculate, can save computing resources when training the neural network model, and improve the efficiency and quality of model training.

[0060] In S140, m sample data are input into a pre-built neural network model, and model parameters of the neural network model are obtained through training, wherein the neural network model includes two layers of convolutional neural network and two layers of multi-layer perceptron.

[0061] In some examples, the first convolutional neural network layer in the two-layer convolutional neural network includes 64 channels, each channel has a kernel size of 7, and the second convolutional neural network layer includes 128 channels, each channel has a kernel size of 7. The first multilayer perceptron layer in the two-layer multilayer perceptron layer reduces the feature dimension of the pixel of each sample data to 64 dimensions, and the second multilayer perceptron layer reduces the 64 dimensions to 1 dimension.

[0062] It is understandable that convolutional neural networks are good at feature extraction in image recognition tasks. Multilayer perceptrons further process features after the convolutional neural network extracts them, helping to reduce the number of parameters and computational complexity while extracting more abstract feature representations.

[0063] The first layer of the convolutional neural network has 64 channels, each channel uses a 7x7 kernel, which can capture local features in the image, such as edges, textures, etc. The second layer of the convolutional neural network has 128 channels, also using a 7x7 kernel, which can capture more complex features, and by increasing the number of channels, the model can learn more feature representations. After the two layers of convolutional neural networks extract features, two layers of multilayer perceptrons are used to further process the features. The first layer of multilayer perceptron reduces the feature dimension to 64 dimensions, which helps to reduce the number of parameters and computational complexity, while extracting more abstract feature representations; the second layer of multilayer perceptron further reduces the 64-dimensional features to 1 dimension, which can be used for the final output in the regression task, that is, predicting the position and surface shape of the object to be measured by the curved contact sensor 100 of the present invention.

[0064] The neural network model that combines a two-layer convolutional neural network and a two-layer multilayer perceptron can learn features at different levels. The model can capture the complex structure in the data, which helps improve the generalization ability of the model and perform well on unseen data. By reducing the feature dimension in the two-layer multilayer perceptron, the model can reduce the risk of overfitting because it limits the capacity of the model so that it cannot be too complex and thus fit too closely to the training data. Reducing the feature dimension can also reduce the number of parameters in subsequent layers, thereby reducing the amount of computation and memory requirements and improving the operating efficiency of the model.

[0065] In S150 , the model parameters of the neural network model are used as calibration parameters of the curved surface contact sensor 100 .

[0066] According to the calibration method of the curved contact sensor 100 of the embodiment of the present invention, the intrinsic parameters and distortion coefficients of the monocular camera can be obtained by utilizing the camera intrinsic parameter and distortion coefficient calibration method; the extrinsic parameters of the monocular camera relative to the contact module 101 can be obtained by utilizing the camera extrinsic parameter calibration method; m predetermined sample data are obtained, and the m sample data are input into a pre-constructed neural network model to train and obtain model parameters of the neural network model, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multilayer perceptron; the model parameters of the neural network model can be used as calibration parameters of the curved contact sensor 100, thereby, the curved contact sensor 100 proposed by the present invention can work normally, so that the curved contact sensor 100 can accurately achieve the detection effect.

[0067] Based on the calibration method of the curved surface contact sensor 100, the present invention further provides a calibration device 10 for the curved surface contact sensor 100. Fig.11 The calibration device 10 of the curved surface contact sensor 100 is described in detail.

[0068] Fig.11 The structure block diagram of the calibration device 10 of the curved surface contact sensor 100 according to the embodiment of the present invention is schematically shown. The curved surface contact sensor 100 includes a monocular camera and a contact module 101.

[0069] The calibration device 10 of the curved contact sensor 100 includes a first determination module 1 , a second determination module 2 , an acquisition module 3 , a training module 4 and a third determination module 5 .

[0070] The first determination module 1 is used to execute a calibration method using camera intrinsic parameters and distortion coefficients to obtain the intrinsic parameters and distortion coefficients of the monocular camera.

[0071] The second determination module 2 is used to execute a camera extrinsic parameter calibration method to obtain the extrinsic parameters of the monocular camera relative to the contact module 101.

[0072] The acquisition module 3 is used to execute the acquisition of predetermined m sample data, wherein each sample data includes a grayscale image of the contact deformation of the contact module 101 captured by the monocular camera after calibration of internal parameters, distortion coefficients and external parameters, and m is an integer greater than or equal to 1.

[0073] The training module 4 is used to input m sample data into a pre-built neural network model to train and obtain model parameters of the neural network model, wherein the neural network model includes two layers of convolutional neural network and two layers of multi-layer perceptron.

[0074] The third determination module 5 is used to use the model parameters of the neural network model as calibration parameters of the curved surface contact sensor 100 .

[0075] According to the calibration device 10 of the curved contact sensor 100 of the embodiment of the present invention, the intrinsic parameters and distortion coefficients of the monocular camera can be obtained by utilizing the camera intrinsic parameter and distortion coefficient calibration method; the extrinsic parameters of the monocular camera relative to the contact module 101 can be obtained by utilizing the camera extrinsic parameter calibration method; m predetermined sample data are obtained, and the m sample data are input into a pre-constructed neural network model, and the model parameters of the neural network model can be trained, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multilayer perceptron; the model parameters of the neural network model can be used as the calibration parameters of the curved contact sensor 100, thereby, the curved contact sensor 100 proposed by the present invention can work normally, so that the curved contact sensor 100 can accurately achieve the detection effect.

[0076] In addition, according to an embodiment of the present invention, any multiple modules among the first determination module 1, the second determination module 2, the acquisition module 3, the training module 4 and the third determination module 5 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0077] According to an embodiment of the present invention, at least one of the first determination module 1, the second determination module 2, the acquisition module 3, the training module 4 and the third determination module 5 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware, or in an appropriate combination of any of them.

[0078] Alternatively, at least one of the first determination module 1, the second determination module 2, the acquisition module 3, the training module 4 and the third determination module 5 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0079] The electronic device according to an embodiment of the present invention includes a processor, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The processor may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include an onboard memory for caching purposes. The processor may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0080] In RAM, various programs and data required for the operation of the electronic device are stored. The processor, ROM and RAM are connected to each other through a bus. The processor performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM and / or RAM. It should be noted that the program can also be stored in one or more memories other than ROM and RAM. The processor can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0081] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface, which is also connected to the bus. The electronic device may also include one or more of the following components connected to the input / output (I / O) interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs communication processing via a network such as the Internet. The drive is also connected to the input / output (I / O) interface as needed. Removable media, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0082] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0083] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include the ROM and / or RAM described above and / or one or more memories other than ROM and RAM.

[0084] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method of the embodiment of the present invention.

[0085] When the computer program is executed by the processor, the above functions defined in the system / device of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0086] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through a communication part, and / or installed from a removable medium. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0087] In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a processor, the above-mentioned functions defined in the system of an embodiment of the present invention are performed. According to an embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0088] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0089] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0091] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A calibration method for a curved surface contact sensor, the curved surface contact sensor comprising a monocular camera and a contact module, characterized in that: The calibration method comprises: Using a camera intrinsic parameter and distortion coefficient calibration method, the intrinsic parameters and distortion coefficients of the monocular camera are obtained; Using a camera extrinsic calibration method, obtaining the extrinsic parameters of the monocular camera relative to the contact module; Acquire m predetermined sample data, wherein each of the sample data comprises a grayscale image of the contact deformation of the contact module captured by the monocular camera after the internal parameter, the distortion coefficient and the external parameter are calibrated, and m is an integer greater than or equal to 1; Inputting the m sample data into a pre-built neural network model, and training to obtain model parameters of the neural network model, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multi-layer perceptron; Using the model parameters of the neural network model as calibration parameters of the curved surface contact sensor; The step of predetermining m sample data includes: Splice and buckle n pre-fabricated first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different arrangement of protrusions, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact areas of the first sample mold and the second sample mold in contact with the contact module are equal, and n is an integer greater than or equal to 1; Using the monocular camera after calibrating the internal parameters, the distortion coefficients and the external parameters, capture preprocessed grayscale images of the second sample mold and each of the first sample molds spliced ​​and buckled on the contact module, and the contact module contact deformation; Removing half of the image captured when the contact module of each of the pre-processed grayscale images is in contact with the second sample mold, and retaining the other half of the image to obtain a cropped image; Splice every two cropped images from n cropped images to get m sample data, where m=n n ; Alternatively, the step of predetermining m sample data includes: Splice and buckle n pre-fabricated first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different arrangement of protrusions, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact areas of the first sample mold and the second sample mold in contact with the contact module are equal, and n is an integer greater than or equal to 1; Using the monocular camera after calibrating the internal parameters, the distortion coefficients and the external parameters, capture preprocessed grayscale images of the second sample mold and each of the first sample molds spliced ​​and buckled on the contact module, and the contact module contact deformation; Removing half of the image captured when the contact module of each of the pre-processed grayscale images is in contact with the second sample mold, and retaining the other half of the image to obtain a cropped image; Splice every two of the n cropped images to get m spliced ​​images, where m=n n ; The m stitched images are respectively subtracted from the reference image to obtain m sample data, wherein the reference image is an image captured when the contact module is in a non-contact state.

2. The calibration method of the curved surface contact sensor according to claim 1, characterized in that: The first convolutional neural network in the two-layer convolutional neural network includes 64 channels, and the kernel size of each channel is 7, and the second convolutional neural network includes 128 channels, and the kernel size of each channel is 7; The first layer of the two-layer multilayer perceptron reduces the feature dimension of each pixel of the sample data to 64 dimensions, and the second layer of the multilayer perceptron reduces the 64 dimensions to 1 dimension.

3. The calibration method of the curved contact sensor according to claim 1, characterized in that: The step of obtaining the external parameters of the monocular camera relative to the contact module by using the camera external parameter calibration method includes: The monocular camera captures a grayscale image of a 1:1 model after inserting the insert k times, wherein the 1:1 model is prefabricated, and the contact module is replaced by the 1:1 model in the curved contact sensor, and the outer surface of the 1:1 model is provided with g insertion holes with known world coordinate positions, which are suitable for inserting the insert, and the layout of the insert inserted each time on the 1:1 model is different, k is an integer greater than or equal to 1, and g is an integer greater than or equal to 1; The external parameters of the monocular camera are calculated according to the world coordinate position of the insert in each grayscale image of the model, the internal parameters of the monocular camera and the distortion coefficient.

4. The calibration method of the curved contact sensor according to claim 1, characterized in that: The camera internal parameter and distortion coefficient calibration method includes a checkerboard calibration method.

5. A calibration device for a curved surface contact sensor, the curved surface contact sensor comprising a monocular camera and a contact module, characterized in that: The calibration device comprises: A first determination module, wherein the first determination module is used to execute a calibration method using camera intrinsic parameters and distortion coefficients to obtain intrinsic parameters and distortion coefficients of the monocular camera; A second determination module, the second determination module is used to execute a camera extrinsic parameter calibration method to obtain an extrinsic parameter of the monocular camera relative to the contact module; An acquisition module, the acquisition module is used to execute acquisition of predetermined m sample data, wherein each of the sample data includes a grayscale image of the contact deformation of the contact module captured by the monocular camera after calibration of the internal parameter, the distortion coefficient and the external parameter, and m is an integer greater than or equal to 1; A training module, the training module is used to input the m sample data into a pre-built neural network model to train and obtain model parameters of the neural network model, wherein the neural network model includes a two-layer convolutional neural network and a two-layer multi-layer perceptron; A third determination module, the third determination module is used to execute the model parameters of the neural network model as calibration parameters of the curved surface contact sensor; Among them, m sample data are predetermined, including: Splice and buckle n pre-fabricated first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different arrangement of protrusions, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact areas of the first sample mold and the second sample mold in contact with the contact module are equal, and n is an integer greater than or equal to 1; Using the monocular camera after calibrating the internal parameters, the distortion coefficients and the external parameters, capture preprocessed grayscale images of the second sample mold and each of the first sample molds spliced ​​and buckled on the contact module, and the contact module contact deformation; Removing half of the image captured when the contact module of each of the pre-processed grayscale images is in contact with the second sample mold, and retaining the other half of the image to obtain a cropped image; Splice every two cropped images from n cropped images to get m sample data, where m=n n ; Alternatively, m sample data are predetermined, including: Splice and buckle n pre-fabricated first sample molds and second sample molds on the contact module, wherein the wall surface of each of the n first sample molds in contact with the contact module has a different arrangement of protrusions, the wall surface of the second sample mold in contact with the contact module is a smooth curved surface, the contact areas of the first sample mold and the second sample mold in contact with the contact module are equal, and n is an integer greater than or equal to 1; Using the monocular camera after calibrating the internal parameters, the distortion coefficients and the external parameters, capture preprocessed grayscale images of the second sample mold and each of the first sample molds spliced ​​and buckled on the contact module, and the contact module contact deformation; Removing half of the image captured when the contact module of each of the pre-processed grayscale images is in contact with the second sample mold, and retaining the other half of the image to obtain a cropped image; Splice every two of the n cropped images to get m spliced ​​images, where m=n n ; The m stitched images are respectively subtracted from the reference image to obtain m sample data, wherein the reference image is an image captured when the contact module is in a non-contact state.

6. An electronic device, characterized in that: include: one or more processors; One or more memories, used to store executable instructions, which, when executed by the processor, implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The storage medium stores executable instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The method comprises a computer program, wherein the computer program comprises one or more executable instructions, and when the executable instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images