Data processing method, terminal device, server, storage medium and program product

Abstract

This disclosure provides a data processing method, terminal device, server, storage medium, and program product, relating to the field of machine learning technology. The method includes: the terminal device acquiring data to be processed collected by a data acquisition module; based on the data to be processed, executing a first processing task using a first processing module in a data processing model to obtain first data; the terminal device sending the first data to a server; the server, based on the first data, executing a second processing task using a second processing module in the aforementioned data processing model to obtain second data; and the server feeding back the second data to the terminal device. The technical solution provided in this disclosure can reduce the amount of data transmitted from the terminal device to the server, reduce data transmission latency, improve the real-time decision-making capabilities of the terminal device, and also avoid the risk of privacy leakage.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning technology, and in particular to a data processing method, terminal device, server, storage medium, and program product. Background Technology

[0002] With the development of artificial intelligence technology, machine learning models have been widely used in the field of image processing.

[0003] Machine learning models typically need to process large amounts of image data, especially when performing tasks on terminal devices (such as autonomous mobile devices), where large amounts of high-resolution image data need to be transmitted to servers for further analysis.

[0004] However, transmitting high-resolution image data to a server typically consumes a large amount of bandwidth and increases data transmission latency, affecting the real-time decision-making capabilities of terminal devices, and also posing a potential risk of privacy leaks. Summary of the Invention

[0005] This disclosure provides a data processing method, terminal device, server, storage medium, and program product, which can reduce the amount of data transmitted from the terminal device to the server, reduce data transmission latency, improve the real-time decision-making capability of the terminal device, and prevent privacy leakage.

[0006] In a first aspect, this disclosure provides a data processing method that can be executed by a data processing device. The data processing device may be a terminal, or a device including a terminal, or a chip (or chip system) or other functional module capable of implementing the functions of a terminal device; for example, the chip or functional module may be disposed within a terminal device.

[0007] The aforementioned terminal device includes a data acquisition module and a first processing module in the data processing model. In some embodiments, the aforementioned method includes:

[0008] Obtain the data to be processed collected by the data acquisition module mentioned above;

[0009] Based on the data to be processed, the first processing module described above is used to execute the first processing task to obtain the first data.

[0010] Send the first data to the server, which includes the second processing module in the above data processing model;

[0011] The server receives second data, which is obtained by the server based on the first data and by performing a second processing task using the second processing module.

[0012] In this embodiment of the disclosure, by deploying the first processing module in the data processing model on the terminal device, the terminal device can process the data to be processed using the first processing module after obtaining the data to be processed, and then send the processed first data to the server for further processing. This reduces the amount of data transmitted from the terminal device to the server, lowers data transmission latency, improves the real-time decision-making capability of the terminal device, and avoids the risk of privacy leakage.

[0013] In one possible implementation, the data acquisition module includes an image acquisition module; the data to be processed includes image data; the first processing task includes an image feature extraction task; and the first data includes feature data corresponding to the image data.

[0014] In the above embodiments, compared to directly sending the image data to the server, sending the feature data as the first data to the server can effectively reduce the amount of data sent by the terminal device to the server.

[0015] In one possible implementation, before executing the first processing task using the first processing module based on the data to be processed, the method further includes:

[0016] The above image data is preprocessed; the preprocessing includes at least one of the following: image normalization, image denoising, image enhancement, image segmentation, or image scaling.

[0017] In the above embodiments, preprocessing the image data helps to improve the image data quality, accelerate the execution speed of subsequent processing tasks, and improve the accuracy and efficiency of the entire data processing model.

[0018] In one possible implementation, the data processing model described above includes a convolutional neural network, which includes n convolutional layers.

[0019] The first processing module includes the first m convolutional layers out of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers out of the n convolutional layers; n and m are both positive integers, and n > m.

[0020] In the above embodiments, by deploying the first processing module in the convolutional neural network on the terminal device and the second processing module on the server, the terminal device can process the data to be processed using the first processing module after obtaining the data to be processed, and then send the processed data to the server, where the server can continue to process the data using the second processing module. This reduces the amount of data transmitted from the terminal device to the server, lowers data transmission latency, and improves the real-time decision-making capability of the terminal device.

[0021] In one possible implementation, the terminal device is an autonomous mobile device; the autonomous mobile device includes any of the following: cleaning robot, companion mobile robot, service mobile robot, industrial autonomous inspection equipment, and security robot.

[0022] In one possible implementation, the data to be processed acquired by the data acquisition module includes:

[0023] The data acquisition module collects image data of obstacles and / or the ground in the environment surrounding the autonomous mobile device.

[0024] In the above embodiments, after acquiring image data of obstacles and / or the ground in the surrounding environment, the autonomous mobile device can first perform preliminary processing on the image data, and then send the processed intermediate data as the first data to the server. This can effectively reduce the amount of data that the autonomous mobile device needs to transmit to the server, saving network bandwidth. Since the direct transmission of the aforementioned image data is reduced, the data upload latency can also be significantly reduced, improving the real-time response capability of the autonomous mobile device in performing tasks.

[0025] Secondly, this disclosure provides a data processing method, which can be executed by a data processing device. The data processing device can be a server, or it can be executed by a chip (or chip system) or other functional module capable of implementing the functions of a server; for example, the chip or functional module is installed in a server. Optionally, the server can be a cloud server or a local server; this disclosure does not limit the scope.

[0026] The server described above includes a second processing module in the data processing model; in some embodiments, the method described above includes:

[0027] Receive first data from a terminal device; the terminal device includes a first processing module in a data processing model, and the first data is obtained by the terminal device based on the data to be processed and by the first processing module performing a first processing task;

[0028] Based on the first data mentioned above, the second processing module is used to execute the second processing task to obtain the second data.

[0029] The second data is fed back to the aforementioned terminal device.

[0030] In one possible implementation, the data processing model described above includes a convolutional neural network, which includes n convolutional layers.

[0031] The first processing module includes the first m convolutional layers out of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers out of the n convolutional layers; n and m are both positive integers, and n > m.

[0032] Thirdly, this disclosure provides a terminal device, including a data acquisition module, a first processing module, a first transmitting module, and a first receiving module in a data processing model; wherein:

[0033] The data acquisition module is used to collect data to be processed.

[0034] The first processing module is used to execute the first processing task based on the acquired data to be processed, and obtain the first data;

[0035] The first sending module is used to send first data to the server, which includes the second processing module in the data processing model;

[0036] The first receiving module is used to receive second data fed back by the server. The second data is obtained by the server based on the first data and by performing a second processing task using the second processing module.

[0037] Fourthly, this disclosure provides a server, including a second processing module, a second sending module, and a second receiving module in a data processing model; wherein:

[0038] The second receiving module is used to receive first data from the terminal device; the terminal device includes a first processing module in the data processing model, and the first data is obtained by the terminal device based on the data to be processed and by the first processing module performing a first processing task;

[0039] The second processing module is used to execute a second processing task based on the first data to obtain the second data;

[0040] The second sending module is used to send second data back to the terminal device.

[0041] Fifthly, this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the data processing method provided in the first aspect or the data processing method provided in the second aspect.

[0042] In a sixth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in the first aspect, or implements the data processing method provided in the second aspect.

[0043] In a seventh aspect, embodiments of this disclosure provide a data processing system, including a terminal device for executing the data processing method provided in the first aspect, and a server for executing the data processing method provided in the second aspect.

[0044] The data processing method, terminal device, server, storage medium, and program product provided in this disclosure, by deploying the first processing module in the data processing model on the terminal device and the second processing module on the server, enable the terminal device to process the data to be processed using the first processing module after obtaining the data to be processed, and then send the processed data to the server, where the server will continue to process the data using the second processing module. This reduces the amount of data transmitted from the terminal device to the server, lowers data transmission latency, improves the real-time decision-making capability of the terminal device, and avoids the risk of privacy leakage. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of an autonomous mobile device provided in an embodiment of this disclosure;

[0046] Figure 2 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this disclosure;

[0047] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this disclosure;

[0048] Figure 4 This is a schematic diagram of the program modules of a terminal device provided in an embodiment of this disclosure;

[0049] Figure 5 This is a schematic diagram of a server program module provided in an embodiment of this disclosure. Detailed Implementation

[0050] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0051] To facilitate a clear description of the technical solutions in the embodiments of this disclosure, the terms "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] In the embodiments of this disclosure, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. For example, "first data" and "second data" are used only to distinguish different data and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily imply that they are different.

[0053] In this disclosure, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc.

[0054] To facilitate a clear description of the technical solutions in the embodiments of this disclosure, some terms and technologies involved in the embodiments of this disclosure are briefly introduced below:

[0055] 1. Terminal equipment

[0056] Terminal devices are input / output devices that connect to servers in a network environment via wired or wireless means. They are distinct from network servers. Examples include personal computers (PCs), mobile phones, and tablets that can connect to the internet.

[0057] In some embodiments, the terminal device may also be an autonomous mobile device, which can refer to an intelligent mobile device that performs predetermined tasks within a set area, including but not limited to: cleaning robots (Robot Vacuum Cleaner, RVC), such as intelligent sweeping robots, intelligent floor scrubbing robots, window cleaning robots, etc.; logistics robots, handling robots, etc.; lawn mowing robots, ice-removing robots, etc.; companion mobile robots, such as intelligent electronic pets, nanny robots; service mobile robots, such as reception robots in hotels, inns, and meeting places; industrial autonomous inspection equipment, such as power inspection robots, intelligent forklifts, etc.; security robots, such as home or commercial intelligent security robots, etc.

[0058] It should be noted that the autonomous mobile devices mentioned above are not limited to the types mentioned above. Any device with autonomous mobility function should be considered an autonomous mobile device as described in the embodiments of this disclosure.

[0059] For example, refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a terminal device, which is an autonomous mobile device, provided in an embodiment of this disclosure.

[0060] Optionally, the aforementioned autonomous mobile device includes: a positioning unit 101, a motion unit 102, an image acquisition module 103, a communication unit 104, a storage unit 105, and a data processing unit 106. In implementation, the terminal device may further include a housing for accommodating the aforementioned units and modules; for example, the image acquisition module 103, communication unit 104, storage unit 105, and data processing unit 106 may be housed within the housing. The housing can be designed in various shapes according to requirements; its material, shape, and size are not limited in this disclosure.

[0061] The positioning unit 101 is connected to the data processing unit 106 and is used to determine the position of the autonomous mobile device. The positioning unit 101 can be one of an inertial measurement unit (IMU), an odometer, a laser rangefinder, or any combination thereof. It can also be an image acquisition module 103 (e.g., a camera) and a positioning module that obtains the real-time position based on the image data (sometimes simply referred to as "image") acquired by the image acquisition module 103 using the Simultaneous Localization and Mapping (SLAM) algorithm.

[0062] The motion unit 102 is connected to the data processing unit 106 and is used to move the entire autonomous mobile device under the control of the data processing unit 106. Optionally, the motion unit 102 includes a motor and a moving component, the motor being used to drive the moving component to move according to the control of the data processing unit. The moving component may be, for example, a wheel, a track, or a multi-legged walking component, etc., and this disclosure does not limit the specific form of the moving component.

[0063] The image acquisition module 103 is connected to the data processing unit 106 and is used to take pictures of the surrounding environment and send the acquired image data to the data processing unit 106. The image acquisition module 103 has photo and / or video recording functions, such as a camera or webcam (e.g., a fisheye camera). The image acquisition module 103 can be a single webcam or multiple independently configured webcams of the same or different types. This disclosure does not limit the number, type, or purpose of the webcams included in the image acquisition module 103.

[0064] The communication unit 104 is connected to the data processing unit 106 and is used to receive or send instructions or data between the terminal device 201 and the server 202. In some embodiments, it may include a first receiving module and a first transmitting module. Optionally, the communication unit can be a wired communication device or a wireless communication device, such as a WiFi module, a General Packet Radio Service (GPRS) module, a Zigbee module, a Bluetooth module, etc.

[0065] Storage unit 105 is connected to data processing unit 106 and can be used to store data.

[0066] The data processing unit 106 can send data or instructions to the server through the communication unit 104, and / or obtain information from the server through the communication unit 104. For example, it can send image data acquired by the image acquisition module 103 to the server 202; or download the calculated map and route planning from the server through the communication unit 104; or complete the positioning and mapping calculations and plan the route locally, while only storing some setting information, historical information and / or map information and data on the server.

[0067] 2. Server

[0068] A server is a device that provides computing power to terminal devices and runs software applications or provides application services in a network environment.

[0069] Optionally, the server may include a cloud server, which is a server architecture built on cloud computing technology. It manages and allocates server resources in a unified manner on a cloud platform in a virtualized form. Users can manage and maintain the virtual server through the cloud platform without needing to concern themselves with the specific operation of the physical server. See the diagram illustrating the connection relationship between server 202 and terminal device 201. Figure 2 For example, server 202 may also include a second receiving module and a second sending module for transmitting data / instructions with terminal device 201.

[0070] 3. Machine learning models

[0071] A machine learning model is a function that maps input data to output. It automatically adjusts its parameters by learning patterns and regularities in the data to minimize prediction error or maximize prediction accuracy. During training, the model continuously adjusts its internal parameters to better fit the training data.

[0072] 4. Deep learning models

[0073] Deep learning models are a type of machine learning model. Based on the structure and principles of neural networks, they learn and make decisions by simulating the workings of the human brain. The principles of deep learning models are primarily based on neural networks and the backpropagation algorithm. A neural network consists of multiple neurons (or nodes) connected by weights and biases to form a hierarchical structure. Information flows from the input layer through hidden layers to the output layer, undergoing linear combinations and nonlinear transformations through the weight connections of each layer to ultimately generate the output. The backpropagation algorithm is the core algorithm for training neural networks. It calculates the gradient of the loss function with respect to the parameters in the network and uses gradient descent to update the parameters, making the network's output closer to the expected target.

[0074] Deep learning models can be classified into various types based on their structure and application scenarios, including: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Generative Adversarial Network (GAN), etc., which are not limited in the embodiments disclosed herein.

[0075] With the development of artificial intelligence technology, machine learning models have been widely used in the field of image processing. Machine learning models typically need to process large amounts of image data, especially when terminal devices (such as autonomous mobile devices) perform tasks, where large amounts of high-resolution image data need to be sent to servers for further analysis. However, sending high-resolution image data to cloud servers usually consumes a lot of bandwidth and causes transmission delays, affecting the real-time decision-making capabilities of terminal devices, and also poses a potential risk of privacy leaks.

[0076] To address the aforementioned technical issues, this disclosure provides a data processing method. This method employs a distributed architecture, deploying a portion of the machine learning model on a terminal device and the remainder on a server. The terminal device can handle front-end data processing and then send the processed intermediate data to the server. This reduces the amount of data sent from the terminal device to the server, lowers data transmission latency, enhances the real-time decision-making capabilities of the terminal device, and also avoids the risk of privacy leaks.

[0077] The technical solutions provided in this disclosure will be described in detail below through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other, and the same or similar content will not be described again in different embodiments.

[0078] In some embodiments, the data processing method provided in this disclosure can be executed by a data processing system. (See also...) Figure 2 , Figure 2 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this disclosure. Figure 2 As shown, the data processing system includes a terminal device 201 and a server 202, and the terminal device 201 and the server 202 communicate via a wireless network.

[0079] The wireless communication between the terminal device 201 and the server 202 can also be simply referred to as "communication". The term "communication" can also be described as "data transmission", "information transmission" or "transmission", and no limitation is made in this embodiment.

[0080] On the terminal side, the data processing system may include a terminal device, or it may include a chip (or chip system) or other functional modules that can implement the functions of the terminal device; for example, the chip or functional module is installed in the terminal device. On the network side, the data processing system may include a server, or it may include a chip (or chip system) or other functional modules that can implement the functions of the server; for example, the chip or functional module is installed in the server. Optionally, the server may be a cloud server or other types of servers; this disclosure does not limit the scope. The following description uses a terminal device and a server as examples.

[0081] In some embodiments, the terminal device includes a data acquisition module for acquiring data to be processed. The data to be processed may be image data, text data, binary data, etc. This disclosure does not limit the type of data to be processed.

[0082] In some implementations, the data acquisition module described above may include an image acquisition module, which can be used to acquire image data.

[0083] Optionally, the image acquisition module described above may include a camera. The camera may be a standard camera, a wide-angle camera, a fisheye camera, or other types of cameras; the acquired image data may be image data under visible light or image data under infrared light, and this embodiment of the disclosure is not limited to any particular type.

[0084] In some embodiments, the terminal device includes a first processing module in the data processing model, and the server includes a second processing module in the data processing model.

[0085] Specifically, the aforementioned data processing model can be a model capable of processing and analyzing data, and it can be divided into at least two parts: a first processing module and a second processing module. The first processing module is deployed in the terminal device and is responsible for initial data processing; the second processing module is deployed in the server and is responsible for further data processing or more complex data analysis, or simply the remaining data processing beyond the initial processing tasks performed by the first processing module. In other words, the first processing module in the terminal device and the second processing module in the server collaborate to achieve the complete functionality of the aforementioned data processing model.

[0086] Optionally, the data processing model described above can be a machine learning model, a deep learning model, or other types of data processing models, and no limitation is imposed in this embodiment.

[0087] For example, refer to Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this disclosure. In some embodiments of this disclosure, the data processing method includes:

[0088] S301. The terminal device acquires the data to be processed collected by the data acquisition module.

[0089] In some implementations, the terminal device collects and acquires data to be processed through its built-in data acquisition module. For example, this data to be processed may be raw, unprocessed information acquired by various sensors, such as image data, sound data, etc.

[0090] S302. The terminal device executes a first processing task based on the data to be processed using the first processing module mentioned above, and obtains the first data.

[0091] In some implementations, after the terminal device acquires the data to be processed, it executes a first processing task using a first processing module (the initial part of the data processing model) deployed on it. This first processing task can perform preliminary processing on the raw data, making the processed data (i.e., the first data) easier to transmit and process.

[0092] S303, The terminal device sends the first data to the server.

[0093] S304. Based on the first data, the server uses the second processing module to execute the second processing task to obtain the second data.

[0094] In some implementations, after receiving the first data, the server executes a second processing task using a second processing module deployed on it (the remaining part of the data processing model excluding the initial part described above). For example, this second processing task may perform further data processing, data analysis, or transformation on the first data to extract useful information or generate a final result.

[0095] S305, The server sends the second data back to the terminal device.

[0096] In some implementations, after receiving the second data, the terminal device may, for example, parse the received second data to extract useful information and perform related operations based on the extracted information.

[0097] In this embodiment of the disclosure, by deploying the first processing module in the data processing model on the terminal device and the second processing module on the server, the terminal device can process the data to be processed using the first processing module after obtaining the data to be processed, and then send the processed data to the server, where the server can continue to process the data using the second processing module. This reduces the amount of data transmitted from the terminal device to the server, lowers data transmission latency, improves the real-time decision-making capability of the terminal device, and avoids the risk of privacy leakage.

[0098] Based on the content described in the above embodiments, in some embodiments of this disclosure, the data to be processed includes image data, the first processing task includes an image feature extraction task, and the first data includes feature data corresponding to the image data.

[0099] For example, a terminal device can acquire raw image data of the surrounding environment collected by a data acquisition module (such as a camera) as data to be processed.

[0100] In some implementations, after acquiring the image data, the terminal device may preprocess the image data before executing the first processing task using the first processing module based on the data to be processed.

[0101] Optionally, the above preprocessing includes at least one of the following:

[0102] Image normalization refers to adjusting the pixel values ​​of image data to a specific range (such as 0 to 1 or -1 to 1) so that subsequent processing algorithms can more easily process and analyze the image data.

[0103] Image denoising refers to the process of removing noise (such as random noise, Gaussian noise, etc.) from image data.

[0104] Image enhancement refers to the process of improving image quality by adjusting attributes such as contrast, brightness, and color.

[0105] Image segmentation: refers to the process of dividing image data into multiple regions or objects, each of which has similar attributes (such as color, texture, etc.).

[0106] Image scaling: refers to the process of adjusting the size of an image (such as its width and height).

[0107] Preprocessing the image data helps improve image data quality, accelerates the execution speed of subsequent processing algorithms, and improves the accuracy and efficiency of the entire data processing model.

[0108] Furthermore, the terminal device can utilize the first processing module to perform the first processing task described above, in order to extract feature data corresponding to the preprocessed image data described above.

[0109] In some implementations, the terminal device may use a first processing module to extract low-dimensional features (or feature maps) from the image data, thereby converting the image data into smaller low-dimensional features, and then uploading these low-dimensional features as the first data to the server.

[0110] The aforementioned low-dimensional features refer to a set of features with lower dimensions extracted from the original high-dimensional data (such as image data). These features can reflect the key information of the original data while reducing the complexity and redundancy of the data.

[0111] For example, the front-end part of a deep learning model (such as CNN, embedding model, encoder, decoder, etc.) is deployed on the terminal device as the first processing module. At this time, the first processing task includes an image feature extraction task to extract low-dimensional features of the image data. Then the first data is the low-dimensional features extracted from the image data that correspond to the image data.

[0112] For example, assuming that the data to be processed collected by the data acquisition module is 640*480 dimension image data in uint8 format, the terminal device uses the first processing module in the deep learning model to perform the image feature extraction task based on the image data, and can extract 64*64 dimension low-dimensional features in uint8 format from the image data. In this embodiment, the low-dimensional features are the first data mentioned above.

[0113] In some implementations, the terminal device can send the aforementioned low-dimensional features as first data to the server for subsequent processing. A comparison shows that the amount of low-dimensional feature data uploaded by the terminal device is significantly less than the amount of original image data (i.e., the data to be processed collected by the aforementioned data acquisition module), thereby reducing the amount of data transmitted from the terminal device to the server.

[0114] Optionally, in some embodiments, before uploading the aforementioned low-dimensional features to the server, the terminal device may further compress these low-dimensional features and send the compressed low-dimensional features as the first data to the server, thereby reducing the time and bandwidth required for the first data transmission. Correspondingly, in some embodiments, if the first data received by the server is compressed low-dimensional features, the received first data may be decompressed first to restore the aforementioned low-dimensional features to their original format.

[0115] When the server receives the first data (original extracted low-dimensional features or compressed low-dimensional features), the second processing module deployed on the server further performs a second processing task on the first data to analyze the information contained in the image data, and feeds back the analysis results (such as the types of obstacles in the image data determined by the deep learning model) as the second data to the terminal device.

[0116] The powerful computing capabilities of the server enable the second processing task to be completed quickly and accurately.

[0117] For example, after receiving the low-dimensional features as the first data, the server can input the received low-dimensional features into the second processing module of the deep learning model deployed in the server. The second processing module can perform specific task processing for different application scenarios (such as obstacle recognition, ground material recognition, etc.) and feed the processing results back to the terminal device as the second data.

[0118] It is understandable that since the first processing module deployed locally on the terminal device and the second processing module deployed on the server belong to the same deep learning model, and the aforementioned low-dimensional features are intermediate data of this deep learning model, deploying the first processing module and the second processing module on the terminal device and the server respectively will not affect the final processing result.

[0119] In some embodiments of this disclosure, the data processing model includes a convolutional neural network, which includes n convolutional layers; the first processing module includes the first m convolutional layers of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers of the n convolutional layers; n and m are both positive integers, and n > m.

[0120] For example, taking the AlexNet convolutional neural network as an example, AlexNet is a deep convolutional neural network that can be applied to image classification. Its network structure contains 5 convolutional layers and 3 fully connected layers. Specifically:

[0121] 1. Convolutional layer:

[0122] The first convolutional layer uses 96 11×11 kernels with a stride of 4 and padding of 0, outputting a feature map of size 55×55.

[0123] The second layer is a pooling layer, which uses a 3×3 max pooling operation with a stride of 2, and outputs a feature map of size 27×27.

[0124] The third convolutional layer uses 256 5×5 convolutional kernels with a stride of 1 and padding of 2, outputting a 27×27 feature map.

[0125] The fourth layer is a pooling layer, which also uses 3×3 max pooling operation with a stride of 2, outputting a feature map of size 13×13.

[0126] The fifth, sixth, and seventh convolutional layers all use 384 or 256 3×3 convolutional kernels with a stride of 1 and padding of 1, outputting a 13×13 feature map.

[0127] 2. Fully connected layer:

[0128] The eighth layer is a fully connected layer containing 4096 neurons, and its input is the output of all the previous convolutional and pooling layers.

[0129] The ninth layer is also a fully connected layer, containing 4096 neurons.

[0130] The eleventh layer is the output layer, which contains 1000 neurons, corresponding to the 1000 categories in the ImageNet dataset.

[0131] In the image processing process, the image passes through 5 convolutional layers in sequence, then is input into 3 fully connected layers, and finally the graph inference result is obtained.

[0132] In some embodiments of this disclosure, the first convolutional layer of the AlexNet network described above can be deployed on a terminal device, while the remaining four convolutional layers and three fully connected layers can be deployed on a server. In this case, n is 5 and m is 1.

[0133] For example, assuming the image data acquired by the terminal device using the data acquisition module has a resolution of 3*227*227, the terminal device can use the first convolutional layer of the locally deployed AlexNet network to extract features from the image data, obtaining feature data with a resolution of 96*27*27. Compared to directly sending the image data acquired by the data acquisition module to the server, sending the aforementioned feature data as the first data to the server can effectively reduce the amount of data sent by the terminal device to the server (the feature data sent to the server as the first data is only 45.27% of its original image data, that is, less than half of the original image data).

[0134] After receiving the first data sent by the terminal device, the server can process the first data using the remaining four convolutional layers and three fully connected layers of the AlexNet network, and feed the processing result back to the terminal device as the second data.

[0135] For example, the YOLO (You Only Look Once) network is used below. The YOLO network is a real-time object detection model that can transform the object detection task into a single regression problem, thereby achieving fast and accurate object detection.

[0136] In some embodiments, the YOLO network described above may consist of a Googlenet network plus four convolutional layers and two fully connected layers.

[0137] In some embodiments of this disclosure, the Google network and the first convolutional layer of the YOLO network described above can be deployed and run on a terminal device, while the remaining three convolutional layers and two fully connected layers can be deployed and run on a server. In this case, n is 4 and m is 1.

[0138] For example, assuming the image data acquired by the terminal device using the data acquisition module has a resolution of 448*448*3, the terminal device can use the GoogleNet network and the first convolutional layer of the locally deployed YOLO network to extract features from the image data (the terminal device can first input the image data into the GoogleNet network, and then input the output data of the GoogleNet network into the first convolutional layer) to obtain feature data with a resolution of 14*14*1024. Compared to directly sending the image data acquired by the data acquisition module to the server, sending this feature data as the first data to the server can effectively reduce the amount of data sent by the terminal device to the server (the feature data sent to the server as the first data is only 1 / 3 of its original image data).

[0139] After receiving the first data sent by the terminal device, the server can process the first data using the remaining three convolutional layers and two fully connected layers of the YOLO network, and feed the processing result back to the terminal device as the second data.

[0140] In this embodiment, since the first data sent by the terminal device to the server is feature data extracted from the image data collected by the terminal device, the amount of first data that the terminal device needs to upload is reduced. Furthermore, since the first data is intermediate data that has only been partially processed by the data processing model and cannot be restored to the original image data to be processed, it can prevent privacy information contained in the original image data from being leaked to the server. Additionally, it ensures that the local computing resource consumption of the terminal device remains at a reasonable level and does not affect the core tasks of the terminal device.

[0141] Based on the content described in the above embodiments, in some embodiments, the terminal device can be an autonomous mobile device. For example, the autonomous mobile device includes any of the following: cleaning robot, companion mobile robot, service mobile robot, industrial autonomous inspection equipment, and security robot.

[0142] In some implementations, the aforementioned autonomous mobile device can acquire image data of obstacles in its surrounding environment collected by the data acquisition module; the aforementioned data processing model can be used to identify the types of obstacles based on the image data of the obstacles.

[0143] Taking a cleaning robot as an example of an autonomous mobile device, the cleaning robot includes an image acquisition module as a data acquisition module, and a first processing module in the data processing model. During operation, the cleaning robot can acquire image data of obstacles in its surrounding environment collected by the image acquisition module, and then use the aforementioned first processing module to extract feature data corresponding to the image data (i.e., perform a first processing task), and send the feature data as first data to the server.

[0144] After receiving the first data sent by the cleaning robot, the server can process the first data (i.e., perform the second processing task) based on the second processing module in the data processing model deployed on the server, identify the types of obstacles, and feed the identification results back to the cleaning robot as the second data.

[0145] In some implementations, after receiving the second data from the server, the cleaning robot can plan its movement path based on the type of obstacle. For example, if it identifies an obstacle such as a floor-standing kitchen cabinet or sofa, the cleaning robot can adjust its trajectory to bypass the furniture and continue cleaning. However, if it identifies an obstacle as a low threshold or a misidentification due to a dark-colored floor, the cleaning robot can proceed directly without avoiding the obstacle or the ground.

[0146] In some implementations, the aforementioned autonomous mobile device can acquire image data of the ground it is located from the data acquisition module; the aforementioned data processing model can be used to identify the material or type of the ground based on the image data.

[0147] Taking a cleaning robot as an example, the cleaning robot includes an image acquisition module and a first processing module in the data processing model.

[0148] Optionally, in some embodiments, the image acquisition module may include an optical flow sensor, which is mounted on the bottom of the cleaning robot and positioned facing the ground, for acquiring image data containing ground texture. The texture includes patterns, lines, and textures on the ground.

[0149] Optionally, in some embodiments, the image acquisition module may also include a camera, which may be positioned facing the ground where the cleaning robot is located, to acquire image data of the ground.

[0150] During its operation, the cleaning robot can acquire image data of the ground it is located on, collected by the image acquisition module. Then, it uses the first processing module to extract feature data corresponding to the image data (i.e., to perform the first processing task) and sends the feature data as the first data to the server.

[0151] After receiving the first data sent by the cleaning robot, the server can process the first data (i.e., perform the second processing task) based on the second processing module in the data processing model deployed on the server, identify the material or type of the ground, and feed the identification result back to the cleaning robot as the second data.

[0152] In some implementations, after receiving the second data from the server, the cleaning robot can perform different functions or working modes according to the material or type of the ground, thereby improving its adaptability to the environment and its intelligence. It can also adjust its movement speed according to the material or type of the ground to ensure work efficiency.

[0153] For example, when the cleaning robot recognizes that it is operating on a low-resistance surface (such as wood flooring, marble, or smooth tile), it can perform a wet mopping function. When it recognizes that it is operating on a high-resistance surface such as carpet, it can avoid the carpet area or stop the wet mopping function and raise the mop with the wet mop. Alternatively, when it recognizes that the surface it is on is a high-resistance surface such as carpet, it can increase the working power of the roller brush and / or the vacuum fan to perform a powerful working mode. When it recognizes that the surface it is on is a low-resistance surface, it can maintain or reduce the working power of the roller brush and / or the vacuum fan to set the working mode to a normal or silent working mode.

[0154] In some implementations, the aforementioned autonomous mobile device may also simultaneously acquire image data of obstacles in its surrounding environment and image data of the ground it is located, as collected by the data acquisition module. See the above embodiments for details, which will not be repeated in this disclosure.

[0155] The data processing method provided in this disclosure, by extracting image features from image data on the autonomous mobile device, can effectively reduce the amount of data sent from the autonomous mobile device to the server, thus saving network bandwidth. Since the autonomous mobile device reduces the direct transmission of high-resolution image data, it can also significantly reduce data upload latency and improve the real-time response capability of the autonomous mobile device in performing tasks. Furthermore, by rationally allocating computing tasks between the autonomous mobile device and the server, it helps to improve the overall performance of the entire system without increasing the burden on the autonomous mobile device.

[0156] The data processing method provided in the embodiments of this disclosure has been described above. The terminal device and server for performing the above data processing method provided in the embodiments of this disclosure are described below.

[0157] In some embodiments, this disclosure provides a terminal device, referring to Figure 4 , Figure 4 This is a schematic diagram of the program modules of a terminal device provided in an embodiment of the present disclosure; in some embodiments, the terminal device 40 includes:

[0158] The data acquisition module 401 is used to acquire data to be processed.

[0159] The first processing module 402 in the data processing model is used to execute a first processing task based on the acquired data to be processed, and obtain the first data.

[0160] The first sending module 403 is used to send first data to the server, which includes the second processing module in the data processing model.

[0161] The first receiving module 404 is used to receive second data fed back by the server. The second data is obtained by the server based on the first data and by performing a second processing task using the second processing module.

[0162] In some embodiments, the data acquisition module 401 includes an image acquisition module; the data to be processed includes image data, the first processing task includes an image feature extraction task; and the first data includes feature data corresponding to the image data.

[0163] In some embodiments, the first processing module 402 is further configured to:

[0164] Before performing the first processing task based on the acquired data to be processed, the image data is preprocessed; the preprocessing includes at least one of the following: image normalization, image denoising, image enhancement, image segmentation, or image scaling.

[0165] In some embodiments, the data processing model described above includes a convolutional neural network, which includes n convolutional layers;

[0166] The first processing module includes the first m convolutional layers out of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers out of the n convolutional layers; n and m are both positive integers, and n > m.

[0167] In some embodiments, the terminal device described above is an autonomous mobile device; the autonomous mobile device includes any one of the following: cleaning robot, companion mobile robot, service mobile robot, industrial autonomous inspection equipment, and security robot.

[0168] In some embodiments, the data acquisition module 401 is specifically used to acquire image data of obstacles and / or the ground in the surrounding environment of the autonomous mobile device.

[0169] The first processing module 402 is specifically used to: acquire image data of obstacles and / or the ground in the surrounding environment of the autonomous mobile device acquired by the image acquisition module 401.

[0170] It is understood that the specific content and beneficial effects performed by the aforementioned terminal device can be referred to the description related to the terminal device in the above data processing method embodiments, and will not be repeated in this disclosure.

[0171] In some embodiments, this disclosure provides a server that can be used to implement the functions of the server described in the above data processing method embodiments.

[0172] Reference Figure 5 , Figure 5This is a schematic diagram of the program modules of a server provided in an embodiment of the present disclosure; in some embodiments, the server 50 includes:

[0173] The second receiving module 501 is used to receive first data from the terminal device; the terminal device includes a first processing module in the data processing model, and the first data is obtained by the terminal device based on the data to be processed and using the first processing module to perform a first processing task.

[0174] The second processing module 502 in the data processing model is used to perform a second processing task based on the first data to obtain the second data.

[0175] The second sending module 503 is used to send second data back to the terminal device.

[0176] In some embodiments, the data processing model includes a convolutional neural network, which includes n convolutional layers; the first processing module includes the first m convolutional layers from the n convolutional layers, and the second processing module includes the remaining nm convolutional layers from the n convolutional layers; n and m are both positive integers, and n > m.

[0177] It is understood that the specific implementation details and beneficial effects of the above-mentioned server can be referred to the server-related descriptions in the above-mentioned data processing method embodiments, and will not be repeated in this disclosure.

[0178] It is understood that the module division in the aforementioned terminal devices and servers is merely a logical functional division. Each function can correspond to a functional module, or two or more functions can be integrated into one functional module. In actual implementation, all or some modules can be integrated into a single physical entity, or they can be distributed across different physical entities. Furthermore, depending on the actual situation, the aforementioned functional modules may be implemented in hardware, software, or a combination of both. Whether a function is executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0179] This disclosure also provides a computer program product comprising: a computer program (also referred to as code or instructions), wherein when the computer program is run, the method executed by the terminal device or the method executed by the server in the above-described embodiments is executed.

[0180] This disclosure also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, the method executed by the terminal device or the method executed by the server in the above-described embodiments is executed.

[0181] This disclosure also provides a data processing system, which includes the aforementioned terminal equipment and server.

[0182] The methods provided in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. This computer program product may include one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions in the embodiments of this disclosure are implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic disk), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0183] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the terminal devices, servers, storage media, and program products described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0185] Furthermore, the functional units / modules in the various embodiments of this disclosure can be integrated into one processing unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated into one unit / module. If the aforementioned functional units / modules are implemented as software functional units / modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or a part thereof, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

Claims

1. A data processing method, characterized in that, Applied in a terminal device, the terminal device including a data acquisition module and a first processing module in a data processing model, the method includes: Acquire the data to be processed collected by the data acquisition module; Based on the data to be processed, the first processing module is used to execute the first processing task to obtain the first data; The first data is sent to a server, wherein the server includes a second processing module in the data processing model; The server receives second data, which is obtained by the server based on the first data and by performing a second processing task using the second processing module.

2. The method according to claim 1, characterized in that, The data acquisition module includes an image acquisition module; the data to be processed includes image data, and the first processing task includes an image feature extraction task; the first data includes feature data corresponding to the image data.

3. The method according to claim 2, characterized in that, Before executing the first processing task using the first processing module based on the data to be processed, the method further includes: The image data is preprocessed; the preprocessing includes at least one of the following: image normalization, image denoising, image enhancement, image segmentation, or image scaling.

4. The method according to any one of claims 1 to 3, characterized in that, The data processing model includes a convolutional neural network, which includes n convolutional layers; The first processing module includes the first m convolutional layers out of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers out of the n convolutional layers; n and m are both positive integers, and n > m.

5. The method according to claim 2 or 3, characterized in that, The terminal device is an autonomous mobile device; the autonomous mobile device includes any one of the following: cleaning robot, companion mobile robot, service mobile robot, industrial autonomous inspection equipment, security robot; the acquisition of the data to be processed collected by the data acquisition module includes: The image acquisition module acquires image data of obstacles and / or the ground in the surrounding environment of the autonomous mobile device.

6. A data processing method, characterized in that, The method is applied in a server, which includes a second processing module in a data processing model; the method includes: Receive first data from a terminal device; the terminal device includes a first processing module in the data processing model, and the first data is obtained by the terminal device based on the data to be processed and by the first processing module performing a first processing task. Based on the first data, the second processing module is used to execute a second processing task to obtain the second data; The second data is fed back to the terminal device.

7. The method according to claim 6, characterized in that, The data processing model includes a convolutional neural network, which includes n convolutional layers; The first processing module includes the first m convolutional layers out of the n convolutional layers, and the second processing module includes the remaining nm convolutional layers out of the n convolutional layers; n and m are both positive integers, and n > m.

8. A terminal device, characterized in that, It includes a data acquisition module, a first processing module, a first sending module, and a first receiving module in the data processing model; The data acquisition module is used to acquire data to be processed; The first processing module is used to execute a first processing task based on the acquired data to be processed, and obtain the first data; The first sending module is used to send the first data to the server, wherein the server includes the second processing module in the data processing model; The first receiving module is used to receive second data fed back by the server. The second data is obtained by the server based on the first data and by performing a second processing task using the second processing module.

9. A server, characterized in that, This includes a second processing module, a second sending module, and a second receiving module in the data processing model; The second receiving module is used to receive first data from the terminal device; the terminal device includes a first processing module in the data processing model, and the first data is obtained by the terminal device based on the data to be processed and by the first processing module performing a first processing task; The second processing module is used to perform a second processing task based on the first data to obtain the second data; The second sending module is used to send the second data back to the terminal device.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data processing method as described in any one of claims 1-7.

11. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the data processing method as described in any one of claims 1-7.

Images