A surface feature recognition method, device and system based on tactile feedback

Abstract

The application discloses a surface feature recognition method, device and system based on tactile feedback, comprising: acquiring an image set; constructing a roughness data set and a texture data set based on the image set; establishing a visual recognition model; training the visual recognition model through the roughness data set and the texture data set to obtain a trained recognition model; identifying a target image according to the trained recognition model to obtain a recognition result, wherein the recognition result comprises a roughness result and a texture result; and outputting a feedback result based on the recognition result. The electro-tactile feedback takes the image recognition result as a driving signal, drives an electric stimulation system device to output an electric pulse stimulation to human skin, constructs an electro-tactile perception induction model, induces a matched tactile perception mode, and realizes tactile perception feedback of object roughness or texture. The application realizes tactile perception distinction of object roughness / texture features and meets human-machine co-integration interaction requirements in various scenes.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and more specifically, to a surface feature recognition method, apparatus, and system based on tactile feedback. Background Technology

[0002] Human-machine collaborative interaction technology refers to the study of intuitive user control of robotic devices and synchronous perceptual feedback from robots to users in autonomously adaptive dynamic environments, achieving bidirectional natural interaction between humans and machines. Existing bidirectional human-machine interaction technologies mainly manifest as user-controlled interfaces on robots, specifically including direct or master-slave control of robot systems through brain-computer interfaces, surface / invasive electromyographic interfaces, and teleoperation technologies for collaborative robots. However, during bidirectional human-machine interaction, the surface roughness and texture features of the controlled object, due to their complexity, are difficult to effectively encode and feedback using simple sensory information such as pressure and displacement.

[0003] It is necessary to combine machine vision and electro-haptic feedback to provide a new method for recognizing object roughness and texture to solve the above-mentioned technical problems. Summary of the Invention

[0004] The purpose of this invention is to provide a surface feature recognition method, device, and system based on tactile feedback, which can meet the needs of human-computer interaction users to recognize the roughness and texture of objects by touch when there is no visual feedback; and provide tactile feedback when there is visual feedback, so that vision and touch work together to improve the human-computer interaction experience.

[0005] To achieve the above objectives, the present invention provides a surface feature recognition method based on tactile feedback, comprising:

[0006] Get the image set;

[0007] Based on the image set, construct a roughness dataset and a texture dataset;

[0008] Establish a visual recognition model;

[0009] The visual recognition model is trained using the roughness dataset and the texture dataset to obtain a trained recognition model.

[0010] The target image is identified based on the trained recognition model to obtain recognition results, wherein the recognition results include roughness results and texture results;

[0011] Feedback results are output based on the recognition results.

[0012] Optionally, constructing the roughness dataset and texture dataset based on the image set includes:

[0013] Based on roughness features, roughness labels are added to the image set to obtain a roughness dataset, wherein the roughness features include protrusion spacing, protrusion shape, and protrusion arrangement.

[0014] Based on texture features, texture labels are added to the image set to obtain a texture dataset, wherein the texture features include solid color texture features and non-solid color texture features.

[0015] Optionally, acquiring the image set includes:

[0016] Obtain an image of the material with a preset roughness as the roughness image;

[0017] Obtain an image of the preset texture material as the texture image;

[0018] The roughness image and the texture image are used as the image set.

[0019] Optionally, the step of adding roughness labels to the image set based on roughness features to obtain a roughness dataset includes:

[0020] The protrusion spacing is determined based on the circumcircle diameter of the protrusion point, wherein the protrusion spacing includes at least five distance levels;

[0021] The protrusion shape is determined based on the protrusion cross-sectional shape of the protrusion point, wherein the protrusion shape includes at least five types;

[0022] The arrangement of the protrusions is determined based on the shape of the line connecting the protrusions, wherein the arrangement of the protrusions includes at least a linear arrangement and a circular arrangement.

[0023] Optionally, constructing the roughness dataset and texture dataset based on the image set further includes:

[0024] Based on the image set, a roughness dataset and a texture dataset are constructed. The roughness dataset includes a roughness training set and a roughness test set, and the texture dataset includes a texture training set and a texture test set. The ratio of the images in the roughness training set and the roughness test set is equal to the ratio of the images in the texture training set and the texture test set.

[0025] Optionally, the visual recognition model includes at least one of an object detection model, an image classification model, and an instance segmentation model, and the step of training the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model includes:

[0026] The visual recognition model is trained using the roughness dataset to obtain a roughness recognition model;

[0027] The visual recognition model is trained using the texture dataset to obtain a texture recognition model.

[0028] Optionally, training the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model further includes:

[0029] Determine whether the roughness feature meets the preset conditions;

[0030] If the roughness feature satisfies the preset condition, the type of the visual recognition model is determined according to the preset model selection strategy, and the visual recognition model is trained to obtain the trained recognition model.

[0031] If the roughness feature does not meet the preset conditions, then any of the visual recognition models is selected and trained to obtain the trained recognition model.

[0032] Optionally, the step of outputting feedback results based on the recognition results includes:

[0033] A roughness feedback signal is determined based on the roughness identification results, wherein the roughness results include protrusion spacing identification results, protrusion shape identification results, and protrusion arrangement identification results, and the roughness feedback signal includes at least one of a first feedback force, a first feedback electrical stimulation, and a first feedback duration;

[0034] A texture feedback signal is determined based on the texture recognition result, wherein the texture feedback signal includes at least one of a second feedback force, a second feedback electrical stimulation, and a second feedback duration;

[0035] The feedback result is determined based on the roughness feedback signal and the texture feedback signal.

[0036] Compared to existing technologies, this invention constructs roughness datasets and texture datasets based on image sets, transforming the surface roughness and texture features of the object to be identified into distinguishable tactile features in machine vision. These features are divided into two independent and unrelated features: roughness and texture. This ensures that the trained recognition model can accurately identify the surface physical features of the object. After the recognition model is trained, the target image is acquired, and two-dimensional results of roughness and texture are obtained, achieving accurate classification of the features and ensuring the accuracy of machine vision-tactile feedback.

[0037] On the other hand, the present invention also provides a surface feature recognition device based on tactile feedback, comprising:

[0038] Image acquisition module, which is used to acquire image sets;

[0039] A dataset construction module, which is used to construct a roughness dataset and a texture dataset based on the image set;

[0040] The model building module is used to build visual recognition models;

[0041] The model training module is used to train the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model.

[0042] The result acquisition module is used to identify the target image according to the trained recognition model and obtain the recognition result, wherein the recognition result includes roughness result and texture result;

[0043] The result output module is used to output feedback results based on the recognition results.

[0044] The surface feature recognition device based on tactile feedback has the same beneficial effects as the surface feature recognition method based on tactile feedback compared with the prior art, and will not be repeated here.

[0045] Thirdly, the present invention also provides a surface feature recognition system based on tactile feedback, including a processor, an image acquisition device, and a feedback triggering device, wherein the processor is used to execute the surface feature recognition method based on tactile feedback as described above, and the surface feature recognition system based on tactile feedback.

[0046] The beneficial effects of the surface feature recognition system based on tactile feedback compared to the prior art are the same as those of the surface feature recognition method based on tactile feedback, and will not be repeated here. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the surface feature recognition method based on tactile feedback according to an embodiment of the present invention.

[0048] Figure 2 This is a detailed flowchart of step S200 of the surface feature recognition method based on tactile feedback according to an embodiment of the present invention.

[0049] Figure 3 This is a detailed flowchart of step S400 of the surface feature recognition method based on tactile feedback according to an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram of the roughness types of the surface feature recognition method based on tactile feedback according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of another roughness type of the surface feature recognition method based on tactile feedback according to an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the texture type of the surface feature recognition method based on tactile feedback according to an embodiment of the present invention. Detailed Implementation

[0053] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0054] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0055] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0056] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0057] like Figure 1 As shown, an embodiment of the present invention provides a method for object roughness / texture perception and recognition that integrates machine vision and tactile feedback, including:

[0058] Step S100: Obtain the image set.

[0059] Because roughness and texture are irregular and their classification is unclear, their description is relatively abstract. Therefore, in this invention, roughness and texture are visualized. Before acquiring the image set, the carriers for roughness and texture are designed to construct roughness datasets and texture datasets. Then, based on the preset carriers, image sets that can represent different roughness and texture are acquired. Specifically, by selecting materials, roughness levels are defined, and images of materials with different roughness levels are acquired as image sets; textures are defined and classified, and images of materials of different categories are acquired as image sets.

[0060] Optionally, the image set includes all graphics and images that can be stored on media.

[0061] In one embodiment, the image set may be acquired by means of a bitmap or vector image acquired by an optical recording device or a scanning device.

[0062] Step S200: Construct a roughness dataset and a texture dataset based on the image set.

[0063] A dataset is constructed using an image set to describe the surface physical characteristics of the objects contained in the image set from two aspects: roughness and texture. Specifically, texture includes the combination of the shape and number of colors of the patterns on the surface of the target objects in the image set, as well as indicators such as the smoothness, transparency, reflectivity, and refractive index of the target object surface, used to describe the patterns or lines on the target object surface; roughness includes the small spacing and the height difference between the peaks and valleys of the target object surface in the image set, used to measure the smoothness of the target object surface.

[0064] For roughness classification, a recognition model trained on a roughness dataset is used for classification; for texture classification, a recognition model trained on a texture dataset is used for classification.

[0065] In one embodiment, the image set consists of multiple images. The roughness dataset and the recognition model dataset belong to the same database. When the dataset is built, images are selected from the database, and the images in the image set are classified. The set of images with greater roughness difference is used as the roughness dataset, and the set of images with more obvious texture difference is used as the texture dataset.

[0066] In another embodiment, all images in the image set are labeled, including labeling the roughness level and texture category of each image, and a roughness dataset and a texture dataset are constructed based on the roughness level and texture category, respectively.

[0067] Step S300: Establish a visual recognition model.

[0068] Specifically, we select deep learning models that are good at recognizing roughness and texture data and have a fast recognition speed to train, so as to ensure timeliness and robustness in actual use.

[0069] In one embodiment, multiple deep learning models are trained simultaneously. After training, the recognition speed and accuracy of the trained models for textures of different roughness and categories are measured. An optimal relationship is established between specific roughness and texture and the deep learning models. For example, after training, the first deep learning model exhibits the fastest speed or highest accuracy in recognizing the first roughness region; while the second deep learning model exhibits the fastest speed or highest accuracy in recognizing the second roughness region. In practical applications, if the target object has a high probability of falling within the first roughness region, the first deep learning model is selected for training and put into use; if the target object has a high probability of falling within the second roughness region, the second deep learning model is selected for training and put into use.

[0070] Step S400: Train the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model.

[0071] Optionally, the visual recognition model is trained using the roughness dataset to obtain a roughness recognition model;

[0072] The visual recognition model is trained using the texture dataset to obtain a texture recognition model.

[0073] In one embodiment, the visual recognition model includes a deep learning model.

[0074] Specifically, a visual recognition model is trained using a roughness dataset to obtain a roughness recognition model, and a texture recognition model is trained using a texture dataset to obtain a texture recognition model.

[0075] In one embodiment, when a certain deep learning model is superior to other deep learning models in both speed and accuracy of roughness and texture recognition, the deep learning model is trained using roughness dataset and texture dataset respectively to obtain a roughness recognition model and a texture recognition model respectively.

[0076] In another embodiment, if a certain deep learning model has a faster and more accurate roughness recognition speed than other deep learning models, then the deep learning model is trained using a roughness dataset to obtain a roughness recognition model; if another deep learning model has a faster and more accurate texture recognition speed than other deep learning models, then the deep learning model is trained using a texture dataset to obtain a texture recognition model.

[0077] In another embodiment, if a certain deep learning model is faster at recognizing roughness than other deep learning models, but another deep learning model is faster at recognizing roughness than the first deep learning model, then a deep learning model is selected for roughness recognition training by a preset method.

[0078] Step S500: Identify the target image according to the trained recognition model and obtain the recognition result, wherein the recognition result includes roughness result and texture result.

[0079] Optionally, the visual recognition model includes at least one of an object detection model, an image classification model, and an instance segmentation model. In one embodiment, the object detection model, classification model, and segmentation model can all achieve the purpose of recognizing roughness and texture based on the image. When recognizing different images, the optimal model can be selected to achieve the best results.

[0080] After the recognition model is trained in step S400, the image of the target is acquired as the target image. The roughness and texture in the target image are identified by the recognition model, and the identified or classified roughness and texture are used as the recognition result.

[0081] In one embodiment, image recognition is performed using a camera. The camera is pointed at the object to be recognized and photographed. Roughness and texture are identified using a recognition model to obtain the recognition result.

[0082] Step S600: Output feedback results based on the recognition results.

[0083] Specifically, feedback results are obtained based on roughness recognition results and texture recognition results.

[0084] In one embodiment, the roughness recognition result and the texture recognition result are fed back separately. For example, when the feedback object is a user, the user is fed back through feedback signals including electrical stimulation to obtain a stimulation signal that matches the roughness result, and then a stimulation signal that matches the texture recognition result is emitted to the user.

[0085] In another embodiment, the roughness recognition results and texture recognition results are fed back in a unified manner. For example, when the feedback is directed to a user, the user is triggered by a feedback signal, including electrical stimulation. The feedback signal includes a stimulation signal that matches the roughness recognition results and texture recognition results.

[0086] In one embodiment, the user obtains visual feedback through a camera or other sensing device while simultaneously triggering tactile feedback by driving a stimulation device based on the recognition results.

[0087] Optionally, the stimulus signal is encoded and mapped to the roughness recognition result and the texture recognition result.

[0088] The recognition model outputs the recognition results as driving signals, which in turn drive the stimulation device to output mapped stimulation pulses. This stimulates the user to elicit distinguishable tactile perception, thus achieving roughness / texture level recognition. The stimulation device has multiple parameters, and combinations of these parameters can induce various tactile patterns. Therefore, the mapping relationship between different driving signals and multiple stimulation parameter sequences is called encoding.

[0089] Optionally, the stimulation device includes an electrical stimulation device and a vibration stimulation device, wherein the electrical stimulation device is used to generate electrical stimulation, and the parameters of the electrical stimulation include current, voltage, and duration; the vibration stimulation device is used to generate vibration stimulation, and the parameters of the vibration stimulation include amplitude, frequency, and duration.

[0090] Optionally, the step of outputting feedback results based on the recognition results includes:

[0091] A roughness feedback signal is determined based on the roughness identification results, wherein the roughness results include protrusion spacing identification results, protrusion shape identification results, and protrusion arrangement identification results, and the roughness feedback signal includes at least one of a first feedback force, a first feedback electrical stimulation, and a first feedback duration;

[0092] A texture feedback signal is determined based on the texture recognition result, wherein the texture feedback signal includes at least one of a second feedback force, a second feedback electrical stimulation, and a second feedback duration;

[0093] The feedback result is determined based on the roughness feedback signal and the texture feedback signal.

[0094] In one embodiment, the stimulation device is controlled by different digital signals, such as high-low level mapping, to output different (electrical or vibrational) stimuli based on roughness recognition results and / or texture recognition results.

[0095] Optionally, constructing the roughness dataset and texture dataset based on the image set includes:

[0096] Based on roughness features, roughness labels are added to the image set to obtain a roughness dataset, wherein the roughness features include protrusion spacing, protrusion shape, and protrusion arrangement.

[0097] Based on texture features, texture labels are added to the image set to obtain a texture dataset, wherein the texture features include solid color texture features and non-solid color texture features.

[0098] In one embodiment, since the meaning of roughness is relatively abstract, roughness is defined by levels, and the roughness levels representing the roughness image set are discretized to ensure accurate recognition and feedback of image recognition results. The roughness features include protrusion spacing (i.e., protrusion density), protrusion shape, and protrusion arrangement. Figure 6 As shown, textures are presented in diverse ways, and their descriptions are relatively abstract. Therefore, textures in the image set are categorized according to their shape and combination, specifically including grid-like, crack-like, spot-like, honeycomb-like, solid-color, scale-like, striped, and swirling shapes. Images with roughness labels are used as the roughness dataset to train the roughness recognition model, while images with texture labels are used as the texture dataset to train the texture recognition model.

[0099] Optionally, textures can be categorized into different shape types for different scenarios.

[0100] Optionally, acquiring the image set includes:

[0101] Obtain an image of the material with a preset roughness as the roughness image;

[0102] Obtain an image of the preset texture material as the texture image;

[0103] The roughness image and the texture image are used as the image set.

[0104] In one embodiment, since the descriptions of roughness and texture are relatively abstract and lack a unified quantification standard, a standardized roughness material is set, and images conforming to the roughness standard are used as roughness images; similarly, a preset texture material is used, and images conforming to the texture standard are used as texture images. An image set composed of various images conforming to different standards is then used to train the recognition model in subsequent steps to obtain a recognition model that can quantify roughness and texture.

[0105] Preferably, such as Figure 4 and Figure 5 As shown, the preset roughness material includes a base platform of 50mm×50mm×10mm and an array of protrusions arranged alternately or randomly with a height of 5mm-10mm, wherein the outer circle diameter of each protrusion is 1-3mm.

[0106] Optionally, such as Figure 2 As shown, the process of adding roughness labels to the image set based on roughness features to obtain a roughness dataset includes:

[0107] Step S210: Determine the protrusion spacing based on the circumcircle diameter of the protrusion point, wherein the protrusion spacing includes at least five distance levels;

[0108] Step S220: Determine the protrusion shape based on the protrusion cross-sectional shape of the protrusion point, wherein the protrusion shape includes at least five types;

[0109] Step S230: Determine the protrusion arrangement according to the shape of the line connecting the equal protrusion points, wherein the protrusion arrangement includes at least a linear arrangement and a circular arrangement.

[0110] Preferably, based on tactile characteristics, the spacing between the protrusions is divided into at least five levels, including 3mm, 4.5mm, 6mm, 7.5mm, and 9mm. Based on the cross-sectional shape of the protrusions, they are divided into at least five cross-sections, including circular, triangular, square, pentagonal, and hexagonal. Based on the arrangement, the protrusions are divided into at least two arrangement methods, including linear and circular arrangements.

[0111] like Figure 4 and Figure 5 As shown, in one embodiment, the cross section perpendicular to the protrusion direction of the protrusion point is used as the cross section, and the cross section perpendicular to the cross section is used as the vertical cross section. The arrangement on the cross section includes a linear arrangement and a circular arrangement. Figure 4 The convex points in the diagram are arranged in a linear pattern. Figure 5 The raised points are arranged in a circular pattern. On the vertical interface, the raised points are arranged in a zigzag or irregular cross pattern, and the height of each raised point can be the same or different.

[0112] In one embodiment, the cross-sectional shape of the protrusion includes the shape that first contacts the protrusion point. The cross-sectional shape of the protrusion point changes with the increase of contact pressure. For example, if the end shape of the protrusion point is round, the contact surface will deform as the contact pressure increases, and the contact surface will gradually change from a point to a circle with an increasingly larger radius. If the end shape of the protrusion point is a cone, the contact surface will gradually increase from a point to a cone-shaped cross-section as the contact pressure increases.

[0113] The texture dataset includes at least eight preset texture categories, including dotted, striped, checkerboard, cracked, swirling, honeycomb, scale-like, and shapeless texture features.

[0114] Optionally, constructing the roughness dataset and texture dataset based on the image set further includes:

[0115] Based on the image set, a roughness dataset and a texture dataset are constructed. The roughness dataset includes a roughness training set and a roughness test set, and the texture dataset includes a texture training set and a texture test set. The ratio of the images in the roughness training set and the roughness test set is equal to the ratio of the images in the texture training set and the texture test set.

[0116] Optionally, after the roughness recognition model and texture recognition model have been trained, it is determined whether the recognition accuracy is greater than a preset accuracy threshold. If yes, the training is complete; otherwise, the recognition model is retrained.

[0117] Preferably, 10% of the images are randomly selected from the roughness dataset and the texture dataset as the test set, and the remaining images are used as the training set.

[0118] Optionally, such as Figure 3 As shown, training the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model further includes:

[0119] Step S410: Determine whether the roughness feature meets the preset conditions;

[0120] Step S420: If the roughness feature satisfies the preset condition, then the type of the visual recognition model is determined according to the preset model selection strategy, and the visual recognition model is trained to obtain the trained recognition model.

[0121] Step S430: If the roughness feature does not meet the preset conditions, select any of the visual recognition models and train them to obtain the trained recognition model.

[0122] In one embodiment, the preset conditions include: the roughness feature belongs to a specific feature, and the specific feature includes roughness features that are recognized with high accuracy and fast recognition speed by some visual recognition models. For example, if the convex cross section is triangular, the first visual recognition model has the fastest recognition speed and the strongest recognition accuracy. Then the roughness feature is a specific feature. When performing feature recognition on the surface of an object with this feature, the first visual recognition model is selected for recognition.

[0123] In one embodiment, after multiple recognition models have been trained, if a particular recognition model is determined to have a superior ability to recognize roughness features, that model is selected to recognize similar features, thereby obtaining a recognition model with higher accuracy and faster recognition speed. If the candidate recognition models are determined to have essentially the same ability to recognize roughness features, any one of them can be selected for training.

[0124] Optionally, a preferred mapping relationship is established between roughness features and recognition models. For example, when a recognition model has a superior ability to recognize a specific roughness feature, a mapping is established between the recognition model and the roughness feature. When the object to be recognized has a high probability of exhibiting the aforementioned roughness feature, the recognition model is used for training and roughness recognition is performed. The mapping table established between the roughness feature / texture feature and the recognition model is used as a preset condition. When the roughness feature or texture feature has a correspondingly superior recognition model, that recognition model is selected for training.

[0125] In one embodiment, the roughness / texture recognition results are fed back in order of rank, from large to small or vice versa, and mapped to the user's perception pattern in a linear / positively correlated manner; the types of tactile perception patterns used by the user to perceive and distinguish object features are set according to needs through parameter encoding settings of electrical stimulation, either with linear intensity enhancement or combined encoding at different locations.

[0126] Another embodiment of the present invention provides a surface feature recognition device based on tactile feedback, comprising:

[0127] Image acquisition module, which is used to acquire image sets;

[0128] A dataset construction module, which is used to construct a roughness dataset and a texture dataset based on the image set;

[0129] The model building module is used to build visual recognition models;

[0130] The model training module is used to train the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model.

[0131] The result acquisition module is used to identify the target image according to the trained recognition model and obtain the recognition result, wherein the recognition result includes roughness result and texture result;

[0132] The result output module is used to output feedback results based on the recognition results.

[0133] The surface feature recognition device based on tactile feedback has the same beneficial effects as the surface feature recognition method based on tactile feedback compared with the prior art, and will not be repeated here.

[0134] Another embodiment of the present invention provides a surface feature recognition system based on tactile feedback, including a processor, an image acquisition device, and a feedback triggering device. The processor is used to execute the surface feature recognition method based on tactile feedback as described above, and the surface feature recognition system based on tactile feedback is described above.

[0135] The beneficial effects of the surface feature recognition system based on tactile feedback compared to the prior art are the same as those of the surface feature recognition method based on tactile feedback, and will not be repeated here.

[0136] Another embodiment of the present invention provides an electronic device, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the surface feature recognition method based on tactile feedback as described above when the computer program is executed.

[0137] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the surface feature recognition method based on tactile feedback as described above.

[0138] Examples of electronic devices that can serve as servers or clients of the present invention, particularly hardware devices applicable to various aspects of the invention, will now be described. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0139] Electronic devices include a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0140] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0141] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0142] While the disclosure is as stated above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this disclosure, and all such changes and modifications will fall within the protection scope of this invention.

Claims

1. A surface feature recognition method based on tactile feedback, characterized in that, include: Get the image set; Based on the image set, a roughness dataset and a texture dataset are constructed, including: Based on roughness features, roughness labels are added to the image set to obtain a roughness dataset, wherein the roughness features include protrusion spacing, protrusion shape, and protrusion arrangement; based on texture features, texture labels are added to the image set to obtain a texture dataset, wherein the texture features include solid color texture features and non-solid color texture features, and the texture label categories include grid-like, crack-like, spot-like, honeycomb-like, solid color, scale-like, stripe-like, and swirl-like. The step of adding roughness labels to the image set based on roughness features to obtain a roughness dataset includes: determining the convexity spacing based on the circumcircle diameter of the convexity points, wherein the convexity spacing includes at least five distance levels; determining the convexity shape based on the convexity cross-sectional shape of the convexity points as a dividing criterion, wherein the convexity shape includes at least five types; and determining the convexity arrangement according to the shape of the lines connecting the convexity points, wherein the convexity arrangement includes at least linear arrangement and circumferential arrangement. Establish a visual recognition model; The visual recognition model is trained using the roughness dataset and the texture dataset to obtain a trained recognition model. The target image is identified based on the trained recognition model to obtain recognition results, wherein the recognition results include roughness results and texture results; Feedback results are output based on the recognition results.

2. The surface feature recognition method based on tactile feedback according to claim 1, characterized in that, The acquired image set includes: Obtain an image of the material with a preset roughness as the roughness image; Obtain an image of the preset texture material as the texture image; The roughness image and the texture image are used as the image set.

3. The surface feature recognition method based on tactile feedback according to claim 1, characterized in that, The construction of the roughness dataset and texture dataset based on the image set also includes: Based on the image set, a roughness dataset and a texture dataset are constructed. The roughness dataset includes a roughness training set and a roughness test set, and the texture dataset includes a texture training set and a texture test set. The ratio of the images in the roughness training set and the roughness test set is equal to the ratio of the images in the texture training set and the texture test set.

4. The surface feature recognition method based on tactile feedback according to claim 1, characterized in that, The visual recognition model includes at least one of an object detection model, an image classification model, and an instance segmentation model. Training the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model includes: The visual recognition model is trained using the roughness dataset to obtain a roughness recognition model; The visual recognition model is trained using the texture dataset to obtain a texture recognition model.

5. The surface feature recognition method based on tactile feedback according to claim 1, characterized in that, The step of training the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model further includes: Determine whether the roughness feature meets the preset conditions; If the roughness feature satisfies the preset condition, the type of the visual recognition model is determined according to the preset model selection strategy, and the visual recognition model is trained to obtain the trained recognition model. If the roughness feature does not meet the preset conditions, then any of the visual recognition models is selected and trained to obtain the trained recognition model.

6. The surface feature recognition method based on tactile feedback according to claim 1, characterized in that, The feedback results output based on the recognition results include: A roughness feedback signal is determined based on the roughness identification results, wherein the roughness identification results include protrusion spacing identification results, protrusion shape identification results, and protrusion arrangement identification results, and the roughness feedback signal includes at least one of a first feedback force, a first feedback electrical stimulation, and a first feedback duration; A texture feedback signal is determined based on the texture recognition result, wherein the texture feedback signal includes at least one of a second feedback force, a second feedback electrical stimulation, and a second feedback duration; The feedback result is determined based on the roughness feedback signal and the texture feedback signal.

7. A surface feature recognition device based on tactile feedback, characterized in that, include: Image acquisition module, which is used to acquire image sets; The dataset construction module is used to construct a roughness dataset and a texture dataset based on the image set, including: adding roughness labels to the image set based on roughness features to obtain a roughness dataset, wherein the roughness features include bulge spacing, bulge shape, and bulge arrangement; and adding texture labels to the image set based on texture features to obtain a texture dataset, wherein the texture features include solid color texture features and non-solid color texture features. The model building module is used to build visual recognition models; The model training module is used to train the visual recognition model using the roughness dataset and the texture dataset to obtain the trained recognition model. The result acquisition module is used to identify the target image according to the trained recognition model and obtain the recognition result, wherein the recognition result includes roughness result and texture result; The result output module is used to output feedback results based on the recognition results; The step of adding roughness labels to the image set based on roughness features to obtain a roughness dataset includes: determining the convexity spacing based on the circumcircle diameter of the convexity point, wherein the convexity spacing includes at least five distance levels; determining the convexity shape based on the convexity cross-sectional shape of the convexity point as a dividing criterion, wherein the convexity shape includes at least five types; and determining the convexity arrangement according to the shape of the line connecting the convexity points, wherein the convexity arrangement includes at least linear arrangement and circumferential arrangement.

8. A surface feature recognition system based on tactile feedback, characterized in that, It includes a processor, an image acquisition device, and a feedback triggering device, wherein the processor is used to execute the surface feature recognition method based on tactile feedback as described in any one of claims 1-6.

Images