Method, device and touch device for determining object recognition model

Abstract

The present application discloses a method, device and touch device for determining an object recognition model. The method includes: obtaining a tactile image and an attribute image obtained by a target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and multiple objects, and the attribute image includes a pressure image, a temperature image and a humidity image; inputting the tactile image and the attribute image into a multi-task learning classification network model for training, extracting feature information of the tactile image through the first branch of the multi-task learning classification network model; inputting the feature information into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object. The present application solves the technical problem of low efficiency in object recognition in the prior art.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things applications, and in particular, to a method, apparatus and touch device for determining an object recognition model. Background Art

[0002] When using smart gloves to touch an object, the data obtained in the prior art only includes pressure images, touch videos, etc. The data collected by the sensors in the smart gloves are processed slowly and cannot be transmitted in real time, and the corresponding object recognition results cannot be obtained quickly.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method, an apparatus and a touch device for determining an object recognition model, so as to at least solve the technical problem of low efficiency in object recognition in the prior art.

[0005] According to one aspect of an embodiment of the present application, a method for determining an object recognition model is provided, comprising: obtaining a tactile image and an attribute image obtained by a target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image; inputting the tactile image and the attribute image into a multi-task learning classification network model for training, and extracting feature information of the tactile image through a first branch of the multi-task learning classification network model; inputting the feature information into a second branch and a third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object.

[0006] Optionally, the touch device includes a friction power generation device, wherein the friction power generation device is used to provide power for sensors in the touch device.

[0007] Optionally, the sensor is used to collect attribute information and tactile images, and the touch device generates the attribute image according to the attribute information, wherein the sensor is a waterproof sensor.

[0008] Optionally, the tactile images and attribute images are input into a multi-task learning classification network model for training, including: determining a data set corresponding to the multi-task learning classification network model, wherein the data set includes tactile images and attribute images; extracting feature information of all tactile images in the data set; and inputting the feature information into a second branch for training to obtain an attribute classification model.

[0009] Optionally, during the process of inputting feature information into the second branch for training, the method further includes: performing a dimensionality reduction operation on the first fully connected layer in the second branch to obtain a first result; and inputting the first result and feature information into the third branch for training to obtain an object recognition model.

[0010] Optionally, the method also includes: when the number of iterations corresponding to the convergence of the attribute classification model is less than the first number of iterations, adjusting the first learning rate of the attribute classification model until the number of iterations corresponding to the convergence of the attribute classification model is greater than or equal to the first number of iterations, wherein the first learning rate represents the convergence state of the attribute classification model; when the number of iterations corresponding to the convergence of the object recognition model is less than the second number of iterations, adjusting the second learning rate of the object recognition model until the number of iterations corresponding to the convergence of the object recognition model is greater than or equal to the second number of iterations, wherein the second learning rate represents the convergence state of the object recognition model.

[0011] Optionally, the tactile image includes a tactile image of a rigid object and a tactile image of a fluid object.

[0012] According to another aspect of an embodiment of the present application, a device for determining an object recognition model is also provided, including: an acquisition module, used to acquire a tactile image and an attribute image obtained by a target object using a touch device to touch an object, wherein the tactile image is an image obtained during the contact between the touch device and the object, and the attribute image includes a pressure image, a temperature image, and a humidity image; an extraction module, used to input the tactile image and the attribute image into a multi-task learning classification network model for training, and extract feature information of the tactile image through the first branch of the multi-task learning classification network model; an input module, used to input the feature information into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object.

[0013] According to another aspect of the embodiments of the present application, a touch device is also provided, including: a sensor, a friction power generation device, a wireless module and a chip, wherein the sensor is used to collect tactile images and attribute images of an object during contact with the object, including a pressure sensor, a temperature sensor and a humidity sensor, and the sensor is implanted in the fabric of the touch device in a woven manner; the friction power generation device is used to provide power to the sensor; the wireless module is used to send the tactile images and attribute images collected by the sensor to the chip; the chip is used to input the tactile images and attribute images into a multi-task learning classification network model for training, and extract feature information of the tactile images through the first branch of the multi-task learning classification network model; the feature information is input into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of object.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned method for determining the object recognition model.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes a computer program, wherein the computer program implements the above-mentioned method for determining the object recognition model when executed by a processor.

[0016] In an embodiment of the present application, tactile images and attribute images are obtained by acquiring a target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image; the tactile image and the attribute image are input into a multi-task learning classification network model for training, and feature information of the tactile image is extracted through the first branch of the multi-task learning classification network model; the feature information is input into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of object, thereby achieving the purpose of using multiple branches of the multi-task learning classification network model to achieve different tasks, thereby achieving the technical effect of identifying multiple attributes of an object, and thus solving the technical problem of low efficiency in object recognition in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for determining an object recognition model according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a method for determining an object recognition model according to an embodiment of the present application;

[0020] Figure 3 is a network structure diagram of a multi-task learning classification network model according to an embodiment of the present application;

[0021] Figure 4 is a structural diagram of a device for determining an object recognition model according to an embodiment of the present application;

[0022] Figure 5is a structural diagram of a touch device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] In the related art, data acquisition only includes pressure images, captured videos, etc.; the use of neural networks supports up to two tasks at the same time; in the selection of glove fabrics, traditional glove fabrics are selected; in the selection of sensors, up to 548 sensors are used; in weight estimation, objects within 60g can be estimated with an accuracy of up to 76%. Therefore, the following problems exist in the related art: 1. The number of sensors is small; 2. The information collected by the sensors is processed slowly and the corresponding feedback cannot be obtained quickly; 3. The number of data sets is small and the recognition accuracy is low. In order to solve the above problems, the embodiments of the present application provide corresponding solutions, which are described in detail below.

[0026] The method for determining an object recognition model of a terminal provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for determining an object recognition model. Figure 1As shown, the computer terminal 10 (or electronic device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0027] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0028] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining the object recognition model in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the method for determining the object recognition model described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).

[0031] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or electronic device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the above-described computer device (or electronic device).

[0032] In the above operating environment, an embodiment of the present application provides an embodiment of a method for determining an object recognition model. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 2 is a flow chart of a method for determining an object recognition model according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0034] Step S202, obtaining a tactile image and an attribute image obtained by the target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image;

[0035] Step S204, inputting the tactile image and the attribute image into a multi-task learning classification network model for training, and extracting feature information of the tactile image through the first branch of the multi-task learning classification network model;

[0036] Step S206, inputting the feature information into the second branch and the third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object.

[0037] In the related art, the processing speed of the collected information is slow, and it is impossible to transmit it in real time and quickly obtain corresponding feedback. In order to improve the feedback speed and shorten the recognition time, in the above steps S202 to S206, by selecting a multi-task learning method, using multiple branches for different classifications, and learning the attribute images and tactile images of multiple objects, the recognition accuracy can be improved and the recognition time can be shortened, thereby achieving the technical effects of object recognition, weight estimation, temperature estimation, and humidity estimation.

[0038] In the above-mentioned method for determining the object recognition model, the touch device includes a friction power generation device, wherein the friction power generation device is used to provide power for the sensor in the touch device, the sensor is used to collect attribute information and tactile images, and the touch device generates an attribute image based on the attribute information, wherein the sensor is a waterproof sensor.

[0039] In the embodiment of the present application, the touch device can be, for example, a smart glove made of graphene material, in which a friction power generation device is embedded in the fabric of the smart glove, and the circuit line is buried in the glove to power the sensor in the smart glove. In the related art, the touch device uses an external power supply, which is poor in operability because the sensor uses a small amount of electricity and a small current can be used to achieve the required amount of electricity. By embedding the friction power generation device in the touch device, the problem of poor operability in the related art can be solved, thereby increasing the convenience of the touch device. At the same time, the waterproof sensor can be woven into the touch device using a knitting method, and the signal transmission and reception can be achieved through a wireless sensor.

[0040] In addition, conventional sensors use porous graphene foam made of carbon-containing materials, which is important for dual sensing properties and scalability, but porous graphene foam itself is hydrophilic, and graphene easily combines with water, thereby changing its molecular structure. In order to make the sensor waterproof, this graphene-form composite material and silicon-based composite material are used to create hydrophobic properties while still retaining the advantages of graphene in sensing temperature. The key to the sensor lies in the combination of materials, especially the point nanomaterials and the corresponding hydrogels, which ultimately form a porous graphene foam composite dual-mode sensor for underwater use.

[0041] In step S204 of the method for determining the above-mentioned object recognition model, the tactile image and the attribute image are input into the multi-task learning classification network model for training, which specifically includes the following steps: determining a data set corresponding to the multi-task learning classification network model, wherein the data set includes tactile images and attribute images; extracting feature information of all tactile images in the data set; and inputting the feature information into the second branch for training to obtain an attribute classification model.

[0042] In the above-mentioned method for determining the object recognition model, during the process of inputting feature information into the second branch for training, the method also includes the following steps: performing a dimensionality reduction operation on the first fully connected layer in the second branch to obtain a first result; and inputting the first result and feature information into the third branch for training to obtain an object recognition model.

[0043] In the embodiment of the present application, a multi-task learning classification network model is first constructed. The multi-task learning classification network model is a lightweight network structure. The specific network structure is as follows: Figure 3 As shown, ReLU is the activation function, Maxpooling layer is the maximum pooling layer, Dropout layer is used to prevent model overfitting, ResNet is the residual network, and the multi-task learning classification network mainly includes three parts: shared network module, image attribute module and object classification module. The shared network module is the first branch mentioned above, which is used to extract the features of the tactile image, and input the learned features of the tactile image into the image attribute module and the object classification module, that is, the second branch and the third branch mentioned above. The network branch of the image attribute module is used to classify the pressure, temperature and humidity images, and judge the tactile image. The network branch of the object classification module is used to classify different objects. Different attributes (pressure, temperature, humidity) of different objects have different tactile images. The second branch and the third branch adopt a cascade operation, and the object classification module divides the first result into 3 groups for further distinction of the types of objects.

[0044] The embodiment of the present application adopts the form of a shared network for feature extraction, which is used to learn the common features of all tasks. The shared network can reduce the number of parameters of the entire network, thereby reducing the amount of calculation and resource requirements. The backbone network (shared network) can be any lightweight network model. The embodiment of the present application takes resnet-50 as an example to illustrate that the resnet network is used and a certain number of layers are cut off as the shared network. The input is a tactile image of a certain size. After multi-layer feature extraction, a feature map of a certain size is formed and sent to the second branch corresponding to the image attribute module.

[0045] The second branch corresponding to the image attribute module is further feature extracted by a certain number of residual modules, and then two fully connected layers are connected to integrate the feature dimensions, and finally the classification calculation of the three attributes is performed through the softmax layer; the third branch corresponding to the object classification module, the present application connects the second and third branches through a cascade operation, and the first fully connected layer in the second branch is reduced in dimension by a reshape operation, and then input into the third branch after resizing. At the same time, the output of the shared network model and the output of the second branch are divided into 3 groups by fusing features of different scales, and each group is convolved by 3 parallel convolutional layers, and the feature extraction results are spliced, and finally classified by the softmax layer. The number of classifications of the third branch is determined by the actual number of objects.

[0046] In the process of training the multi-task learning classification network model, the data set used is a tactile video frame formed each time different objects come into contact with the touch device, which is collected at a fixed sampling frequency to obtain a tactile image during the contact process. Since the tactile image is obtained under different attributes, the attribute image of the object can be obtained according to the different attributes of the object. The attribute image includes the attribute information of the object, and the attribute image includes a pressure image, a temperature image, and a humidity image. In an optional embodiment, the attribute information of the object can be obtained by annotating the tactile image, and a label file is formed by the annotation, and the label file contains attribute information such as the object type, pressure, temperature, and humidity.

[0047] In the embodiment of the present application, based on the above description, a segmented training method is adopted to train the multi-task learning classification network model, and the specific process is as follows:

[0048] StageA: Train the second branch and use the collected tactile images and attribute images to train the classification of different attributes such as pressure, temperature, and humidity to obtain the classification model of this stage, namely the attribute classification model, where the attribute image can be replaced by the annotation file corresponding to the tactile image.

[0049] StageB: Train the third branch, using the attribute classification model obtained in StageA as the pre-training model, and using the same tactile images and attribute images, or tactile images and corresponding object label files for training to obtain an object recognition model.

[0050] In the above-mentioned method for determining the object recognition model, the method also includes the following steps: when the number of iterations corresponding to the convergence of the attribute classification model is less than the first number of iterations, adjusting the first learning rate of the attribute classification model until the number of iterations corresponding to the convergence of the attribute classification model is greater than or equal to the first number of iterations, wherein the first learning rate represents the convergence state of the attribute classification model; when the number of iterations corresponding to the convergence of the object recognition model is less than the second number of iterations, adjusting the second learning rate of the object recognition model until the number of iterations corresponding to the convergence of the object recognition model is greater than or equal to the second number of iterations, wherein the second learning rate represents the convergence state of the object recognition model.

[0051] In the embodiment of the present application, the same optimization strategy is used for the above-mentioned Stage A and Stage B. When the number of iterations is less than the preset number of iterations, the Adam optimization method is used to update or adjust the learning rate of the network model until the network loss converges or exceeds the set number of iterations. Since the two-stage tasks are different, the embodiment of the present application uses different learning rates for the training process of the two branches. For the direction of fast update, a smaller learning rate is configured, and for the direction of slow update, a larger learning rate is required.

[0052] Through continuous learning and training, the final object recognition model is obtained. Users can quickly identify objects and obtain relevant parameters such as weight, temperature, and humidity by touching objects with their touch devices.

[0053] In the above-mentioned method for determining the object recognition model, the largest pressure data set currently obtained by touching an object with a touch device is obtained by touching 26 objects. The number of data sets determines the performance of the recognition network. Since the existing data sets are small in data volume and type, the recognition rate of objects is poor. In order to improve the recognition accuracy, it is necessary to increase training data. Therefore, the tactile images in the embodiment of the present application include tactile images of rigid objects and tactile images of fluid objects. By increasing the types and number of data sets, the object recognition accuracy is improved. It should be noted that the fluid objects in the embodiment of the present application include but are not limited to various liquids such as water.

[0054] The embodiment of the present application can learn to identify a single object, identify the object, estimate its weight, identify its temperature and humidity, and complete tactile feedback. First, in order to make the tactile device more likely to contact the hand, the sensor is implanted into the fabric of the tactile device by weaving; secondly, when the smart fabric performs object recognition, the construction of the data set needs to try a large number of objects, which at least include rigid objects and fluid objects. The tactile device touches different objects multiple times. Each time the object contacts the touch device, the sensor will send back the attribute image of the object and the recorded tactile video. The tactile video is composed of different tactile images formed when the tactile device contacts the object. The attribute image returned by the same touch object and each frame of the tactile image of the tactile video are input into the multi-task learning network; finally, in the selection of the neural network, a multi-task processing network is used for feature extraction and task learning, and the obtained information is quickly fed back. The feedback results include object recognition, weight estimation, temperature estimation, etc.

[0055] The method for determining the object recognition model provided in the embodiment of the present application can be applied to the following scenarios: 1. Tactile feedback of prostheses. Providing tactile feedback for prostheses can help alleviate the perception of pain of missing limbs, increase the recognition of the prosthesis as a part of the body, and reduce the cognitive stress involved in controlling the prosthesis through more natural operations; 2. Robots and mechanical arms. Tactile feedback plays a vital role in controlling the movement and force of the hands. Therefore, the application of the object recognition model in the present application can obtain tactile feedback in a timely manner and achieve stable gripping force.

[0056] Figure 4 is a structural diagram of a device for determining an object recognition model according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:

[0057] An acquisition module 402 is used to acquire a tactile image and an attribute image obtained by the target object using a touch device to touch an object, wherein the tactile image is an image obtained during the contact between the touch device and the object, and the attribute image includes a pressure image, a temperature image, and a humidity image;

[0058] An extraction module 404 is used to input the tactile image and the attribute image into a multi-task learning classification network model for training, and extract feature information of the tactile image through a first branch of the multi-task learning classification network model;

[0059] The input module 406 is used to input feature information into the second branch and the third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify attribute images and the third branch is used to classify the types of objects.

[0060] In the extraction module in the above-mentioned object recognition model determination device, the tactile image and the attribute image are input into the multi-task learning classification network model for training, which specifically includes the following process: determining a data set corresponding to the multi-task learning classification network model, wherein the data set includes tactile images and attribute images; extracting feature information of all tactile images in the data set; inputting the feature information into the second branch for training to obtain an attribute classification model.

[0061] In the extraction module in the above-mentioned object recognition model determination device, during the process of inputting feature information into the second branch for training, the module also includes the following process: after performing a dimensionality reduction operation on the first fully connected layer in the second branch, a first result is obtained; the first result and feature information are input into the third branch for training to obtain an object recognition model.

[0062] In the input module in the above-mentioned object recognition model determination device, the module includes the following process: when the number of iterations corresponding to the convergence of the attribute classification model is less than the first number of iterations, adjusting the first learning rate of the attribute classification model until the number of iterations corresponding to the convergence of the attribute classification model is greater than or equal to the first number of iterations, wherein the first learning rate represents the convergence state of the attribute classification model; when the number of iterations corresponding to the convergence of the object recognition model is less than the second number of iterations, adjusting the second learning rate of the object recognition model until the number of iterations corresponding to the convergence of the object recognition model is greater than or equal to the second number of iterations, wherein the second learning rate represents the convergence state of the object recognition model.

[0063] It should be noted that Figure 4 The object recognition model determination device shown is used to perform Figure 2 The method for determining the object recognition model shown in the figure, therefore the relevant explanations in the above-mentioned method for determining the object recognition model are also applicable to the device for determining the object recognition model, and will not be repeated here.

[0064] Figure 5 is a structural diagram of a touch device according to an embodiment of the present application, such as Figure 5As shown, the touch device 500 includes: a sensor 502, a friction power generation device 504, a wireless module 506 and a chip 508, wherein the sensor is used to collect tactile images and attribute images of the object during contact with the object, including a pressure sensor, a temperature sensor and a humidity sensor, and the sensor is woven into the fabric of the touch device; the friction power generation device is used to provide power for the sensor; the wireless module is used to send the tactile images and attribute images collected by the sensor to the chip; the chip is used to input the tactile images and attribute images into a multi-task learning classification network model for training, and extract feature information of the tactile images through the first branch of the multi-task learning classification network model; the feature information is input into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object.

[0065] It should be noted that Figure 5 The touch device shown is used to perform Figure 2 The method for determining the object recognition model shown in the figure, therefore the relevant explanations in the above method for determining the object recognition model are also applicable to the touch device and will not be repeated here.

[0066] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following method for determining an object recognition model: obtaining a tactile image and an attribute image obtained by a target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image; inputting the tactile image and the attribute image into a multi-task learning classification network model for training, and extracting feature information of the tactile image through the first branch of the multi-task learning classification network model; inputting the feature information into the second branch and the third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of object.

[0067] An embodiment of the present application also provides a computer program product, which includes a computer program, wherein the computer program implements the following method for determining an object recognition model when executed by a processor: obtaining a tactile image and an attribute image obtained by a target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image; inputting the tactile image and the attribute image into a multi-task learning classification network model for training, and extracting feature information of the tactile image through the first branch of the multi-task learning classification network model; inputting the feature information into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of object.

[0068] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0069] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0071] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0072] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0073] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0074] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining an object recognition model, characterized in that: include: Acquire a tactile image and an attribute image obtained by the target object using a touch device to touch multiple objects, wherein the tactile image is an image obtained during the contact between the touch device and the multiple objects, and the attribute image includes a pressure image, a temperature image, and a humidity image; Inputting the tactile image and the attribute image into a multi-task learning classification network model for training, and extracting feature information of the tactile image through a first branch of the multi-task learning classification network model; Inputting the feature information into the second branch and the third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object; Inputting the tactile image and the attribute image into a multi-task learning classification network model for training, including: determining a data set corresponding to the multi-task learning classification network model, wherein the data set includes the tactile image and the attribute image; extracting feature information of all tactile images in the data set; inputting the feature information into the second branch for training to obtain an attribute classification model; In the process of inputting the feature information into the second branch for training, the method further includes: performing a dimensionality reduction operation on the first fully connected layer in the second branch to obtain a first result; and inputting the first result and the feature information into the third branch for training to obtain the object recognition model.

2. The method according to claim 1, characterized in that The touch device includes a friction power generation device, wherein the friction power generation device is used to provide power for the sensor in the touch device.

3. The method according to claim 2, characterized in that The sensor is used to collect attribute information and the tactile image, and the touch device generates the attribute image according to the attribute information, wherein the sensor is a waterproof sensor.

4. The method according to claim 1, characterized in that: The method further comprises: When the number of iterations corresponding to the convergence of the attribute classification model is less than the first number of iterations, adjusting the first learning rate of the attribute classification model until the number of iterations corresponding to the convergence of the attribute classification model is greater than or equal to the first number of iterations, wherein the first learning rate represents the convergence state of the attribute classification model; When the number of iterations corresponding to the convergence of the object recognition model is less than the second number of iterations, adjust the second learning rate of the object recognition model until the number of iterations corresponding to the convergence of the object recognition model is greater than or equal to the second number of iterations, wherein the second learning rate represents the convergence state of the object recognition model.

5. The method according to claim 1, characterized in that The tactile images include tactile images of rigid objects and tactile images of fluid objects.

6. A device for determining an object recognition model, characterized in that: include: An acquisition module, used to acquire a tactile image and an attribute image obtained by a target object using a touch device to touch an object, wherein the tactile image is an image obtained during the contact between the touch device and the object, and the attribute image includes a pressure image, a temperature image, and a humidity image; an extraction module, configured to input the tactile image and the attribute image into a multi-task learning classification network model for training, and extract feature information of the tactile image through a first branch of the multi-task learning classification network model; An input module, used to input the feature information into the second branch and the third branch in the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object; The extraction module is also used to determine a data set corresponding to the multi-task learning classification network model, wherein the data set includes the tactile images and the attribute images; extract feature information of all tactile images in the data set; input the feature information into the second branch for training to obtain an attribute classification model; The extraction module is also used to obtain a first result after performing a dimensionality reduction operation on the first fully connected layer in the second branch; and input the first result and the feature information into the third branch for training to obtain the object recognition model.

7. A touch device, characterized in that: include: Sensors, triboelectric devices, wireless modules and chips, among which, The sensor is used to collect the tactile image and attribute image of the object during the contact with the object, including a pressure sensor, a temperature sensor and a humidity sensor, and the sensor is embedded in the fabric of the touch device in a woven manner; The friction power generation device is used to provide power for the sensor; The wireless module is used to send the tactile image and the attribute image collected by the sensor to the chip; The chip is used to input the tactile image and the attribute image into a multi-task learning classification network model for training, extract feature information of the tactile image through the first branch of the multi-task learning classification network model; input the feature information into the second branch and the third branch of the multi-task learning classification network to obtain an object recognition model, wherein the second branch is used to classify the attribute image, and the third branch is used to classify the type of the object; input the tactile image and the attribute image into the multi-task learning classification network model for training, including: determining a data set corresponding to the multi-task learning classification network model, wherein the data set includes the tactile image and the attribute image; extracting feature information of all tactile images in the data set; inputting the feature information into the second branch for training to obtain an attribute classification model; the process of inputting the feature information into the second branch for training also includes: After performing a dimensionality reduction operation on the first fully connected layer in the second branch, a first result is obtained; the first result and the feature information are input into the third branch for training to obtain the object recognition model.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method for determining the object recognition model according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program, when executed by a processor, implements the method for determining the object recognition model according to any one of claims 1 to 5.

Images