Detergent control method and device based on deep learning, washing machine and medium

Abstract

The invention relates to a detergent control method and device based on deep learning, a washing machine and a medium. The method comprises the following steps: acquiring washing characteristics of a detergent and acquiring state characteristics of washings; constructing feature vectors based on the washing features of the detergent and the state features of the washings through a pre-trained deep reinforcement learning model; feature analysis is carried out according to the feature vectors through a pre-trained deep reinforcement learning model, target vectors are determined, and each element in the target vectors corresponds to a target estimated value of one detergent putting strategy; and based on the target estimation value of the target vector, selecting a detergent putting strategy corresponding to the maximum target estimation value as an optimal washing machine putting strategy to perform washing machine putting quantity control. According to the method, the parameters of the detergent and the washings are obtained, feature analysis is performed in combination with deep reinforcement learning, and the optimal detergent putting strategy is selected to perform precise control on the putting of the washing machine, so that the putting flexibility and accuracy of the washing machine are improved, and a better washing effect can be achieved.

Description

Technical Field

[0001] The present application relates to the field of household appliance control technology, and in particular to a detergent control method, device, washing machine and medium based on deep learning. Background Art

[0002] In modern family life, washing machines have become an indispensable household appliance. With the advancement of technology, consumers have higher and higher requirements for intelligent and automated washing machines. Traditional washing machines require users to manually add detergent, which is not only cumbersome to operate, but also difficult to accurately add detergent according to the specific conditions of the clothes, resulting in detergent waste or poor washing results, for example, according to the weight, degree of dirtiness, fabric quality, etc. of the clothes.

[0003] In the prior art, although some washing machines have begun to adopt automatic dispensing technology, these technologies mostly rely on simple sensors and preset programs, can only achieve quantitative detergent dispensing, lack flexibility and adaptability, and cannot achieve accurate detergent dispensing under diverse washing needs. It can be seen that in the prior art, there are still problems of poor flexibility and low control accuracy in the automatic dispensing of detergents. Summary of the invention

[0004] The present application provides a detergent control method, device, washing machine and medium based on deep learning to solve the technical problems of poor flexibility and low control accuracy in the automatic delivery of detergent in the above-mentioned prior art.

[0005] According to one aspect of an embodiment of the present application, the present application provides a detergent control method based on deep learning, the method comprising: obtaining washing characteristics of the detergent and obtaining state characteristics of the laundry; constructing a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model; performing feature analysis based on the feature vector through the pre-trained deep reinforcement learning model to determine a target vector, wherein each element in the target vector corresponds to a target estimated value of a detergent delivery strategy; based on the target estimated value of the detergent delivery strategy in each element of the target vector, selecting the detergent delivery strategy corresponding to the maximum target estimated value as the optimal washing machine delivery strategy to control the washing machine delivery amount.

[0006] Optionally, obtaining the washing characteristics of the detergent includes: collecting an image of the detergent, preprocessing the image of the detergent through an optical character recognition model to obtain a preprocessed image; performing text recognition and conversion on the preprocessed image to obtain detergent character information; and performing text parsing and information extraction based on the detergent character information to obtain the washing characteristics of the detergent.

[0007] Optionally, the state characteristics include stain characteristics, and obtaining the state characteristics of the laundry includes: collecting laundry images of the laundry in multiple dimensions; performing feature recognition on the laundry images of each dimension through a pre-trained image recognition model to extract laundry characteristics; and extracting the stain characteristics of the laundry based on the laundry characteristics.

[0008] Optionally, the state characteristics also include laundry weight characteristics, dirtiness characteristics and fabric characteristics, and the obtaining of laundry state characteristics also includes: when the laundry is placed in the washing machine, respectively collecting the laundry weight characteristics, dirtiness parameters and fabric parameters of the laundry based on different sensors; performing data preprocessing on the dirtiness parameters and fabric parameters, and performing primary feature extraction based on the preprocessed data; performing high-level feature extraction on the features obtained from the primary feature extraction to obtain high-level dirtiness features and high-level fabric features; performing feature selection and dimensionality reduction on the features obtained from the high-level feature extraction to respectively obtain the dirtiness features and the fabric features.

[0009] Optionally, the pre-trained deep reinforcement learning model performs feature analysis based on the feature vector to determine the target vector, including: receiving the feature vector through the input layer of the pre-trained deep reinforcement learning model, wherein the number of neurons in the input layer is consistent with the dimension of the feature vector; performing a nonlinear transformation on the feature vector based on the hidden layer of the pre-trained deep reinforcement learning model and outputting it to the output layer, and outputting the target vector based on the output layer.

[0010] Optionally, after the hidden layer based on the pre-trained deep reinforcement learning model performs a nonlinear transformation on the feature vector and outputs it to the output layer, and after the target vector is output based on the output layer, the method further includes: predicting an estimated value of the current state based on the current state and the detergent dosage strategy based on an online network in the pre-trained deep reinforcement learning model, and predicting a target estimated value of the next state based on the target network in the pre-trained deep reinforcement learning model; calculating a loss based on the estimated value of the current state and the target estimated value; and updating the network weights in the pre-trained deep reinforcement learning model through back propagation based on the loss.

[0011] Optionally, the method further includes: obtaining execution result data of detergent dosage control based on the optimal detergent dosage strategy; and constructing a new feature vector based on the execution result data, the washing characteristics of the detergent and the state characteristics of the laundry to optimize the pre-trained deep reinforcement learning model.

[0012] According to another aspect of an embodiment of the present application, the present application provides a detergent control device based on deep learning, the device comprising: a data acquisition module, used to acquire the washing characteristics of the detergent and the state characteristics of the laundry; a vector construction module, used to construct a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model; a feature extraction module, used to perform feature analysis according to the feature vector through the pre-trained deep reinforcement learning model to determine a target vector, each element in the target vector corresponds to a target estimated value of a detergent delivery strategy; a control module, used to select the detergent delivery strategy corresponding to the maximum target estimated value as the optimal washing machine delivery strategy based on the target estimated value of the detergent delivery strategy in each element of the target vector to control the delivery amount of the washing machine.

[0013] According to another aspect of an embodiment of the present application, the present application provides a washing machine, which controls the automatic addition of detergent according to the above-mentioned deep learning-based detergent control method.

[0014] According to another aspect of an embodiment of the present application, the present application provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the steps of the detergent control method based on deep learning.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0016] The present application provides a detergent control method based on deep learning, which is applicable to the intelligent input control scenario of detergents in washing machines and other similar washing equipment. By acquiring the washing characteristics of the detergent and the state characteristics of the laundry to construct a feature vector, the accuracy and dimension of the data parameters of the pre-trained deep reinforcement learning model can be improved, which is more conducive to the pre-trained deep reinforcement learning model to perform more accurate feature extraction and recognition through continuous trial and error and feedback learning in a complex environment, so as to select the optimal detergent delivery strategy, and realize more flexible and accurate automatic delivery control of the detergent based on the optimal detergent delivery strategy, ensuring that the laundry is fully cleaned while avoiding detergent waste, and is conducive to reducing pollution to the environment and reducing water waste. It can also meet consumers' needs for convenience and comfort by improving the intelligence level of washing machines. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A schematic diagram of a hardware environment of an optional detergent control method based on deep learning provided according to an embodiment of the present application;

[0020] Figure 2 A schematic flow chart of an optional detergent control method based on deep learning provided according to an embodiment of the present application;

[0021] Figure 3 This is a flow chart of an optional step S202 provided according to an embodiment of the present application;

[0022] Figure 4 A schematic flow chart of another optional detergent control method based on deep learning provided according to an embodiment of the present application;

[0023] Figure 5 This is a schematic structural diagram of an optional detergent control device based on deep learning provided according to an embodiment of the present application;

[0024] Figure 6 A schematic diagram of an optional computer device structure provided for an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0026] In order to solve the problems mentioned in the background technology, according to one aspect of an embodiment of the present application, an embodiment of a detergent control method based on deep learning is provided.

[0027] like Figure 1 As shown, the detergent control method based on deep learning is applicable to Figure 1 In the hardware system environment shown, the system architecture of the hardware system environment includes a terminal device 101, a server 103 and a washing machine 105. The terminal device 101 includes a client, and the client includes a smart home system APP.

[0028] The smart home system can be controlled on the terminal device 101, and the washing machine can be controlled by using the server and the network. The server 103 is connected to the terminal device 101 and the washing machine 105 through the network, and can be used to provide services for the terminal or the client installed on the terminal and the washing machine 105, including processing data requests and instructions from the terminal device 101 and the washing machine 105 through the server 103 to achieve communication and data exchange between the two. A database 107 can be set on the server or independently of the server to provide data storage services for the server 103. Among them, the network can include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The washing machine can include a washing machine and cleaning equipment similar to a washing machine, such as a shoe washing machine, etc.

[0029] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc. The server 103 may be a server providing various services, for example, a background server providing support for the web pages displayed on the terminal device 101.

[0030] Among them, the smart home system integrates facilities related to home life by using integrated wiring technology, network communication technology, security technology, automatic control technology, and audio and video technology to build an efficient management system for residential facilities and family schedule affairs. Through the smart home system, users can remotely control operations such as starting, pausing, and shutting down the washing machine. In addition, the system can also automatically adjust parameters such as washing mode, water temperature, and speed according to the user's settings and the actual situation of the clothes to achieve a more intelligent washing process. The smart home system can monitor the operating status of the washing machine in real time, including washing progress, remaining time, etc. This information can be fed back to the user in real time through the terminal device 101, so that the user can keep track of the use of the washing machine at any time.

[0031] Furthermore, linkage rules can be set in the smart home system. For example, after the user puts in the clothes in the early stage of the operation, the user's mobile phone starts the washing mode of the washing machine by operating the smart home system; when the washing machine completes the washing, a notification is sent to the user's mobile phone through the smart home system. At the same time, the user interface module of the washing machine connected to the user terminal device can display the current status of the washing machine, the amount of detergent, and other information, and provide a user operation interface. The user can set the washing mode and view the washing progress through the user interface module.

[0032] It should be noted that the detergent control method based on deep learning provided in the embodiment of the present application is generally executed by a server and / or a terminal device, and accordingly, the detergent control device based on deep learning is generally set in the server / terminal device. And it should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0033] like Figure 2 As shown, Figure 2 A flowchart of a detergent control method based on deep learning provided by an embodiment of the present invention. Taking the detergent control method based on deep learning executed by a server as an example, the detergent control method based on deep learning includes the following steps:

[0034] Step S202, obtaining washing characteristics of the detergent and obtaining state characteristics of the laundry.

[0035] In this embodiment, the washing characteristics of the detergent can be obtained by a device such as a camera or a scanner, wherein the detergent can be washing powder, washing liquid, washing beads, etc. The camera or scanner can be installed on a washing machine. The following embodiments are described using a washing machine, for example, installed at a preset position on the washing machine. The washing machine is a smart washing machine. The above washing characteristics may include but are not limited to brand name, product type, main ingredients, concentration identification of each ingredient, recommended dosage table, etc.

[0036] In this embodiment, the state characteristics of the laundry can be collected by the camera and the sensor, wherein the laundry is described in the embodiment by taking clothes as an example. The above state characteristics may include stains, weight, degree of dirtiness, fabric quality, etc. of the clothes. For example, the stains of the clothes are collected by the camera, the weight of the clothes is collected by the weight sensor, the degree of dirtiness of the clothes is collected by the turbidity sensor, and the fabric quality of the clothes is collected by the material recognition sensor.

[0037] Step S204: constructing a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model.

[0038] Among them, Deep Reinforcement Learning (DRL) is an artificial intelligence technology that combines deep learning and reinforcement learning. It can combine the representation ability of deep learning with the decision-making ability of reinforcement learning to achieve higher performance. The characteristics of deep reinforcement learning include: optimizing decision strategies through automatic exploration and learning, without pre-defined rules or guidance, and finding the optimal decision strategy through continuous interaction with the environment through trial and error; it can handle high-dimensional and complex environmental states, and can extract high-level features from a large amount of input data to make more accurate decisions; it has the ability of adaptive learning, does not require artificially designed features or rules, and can automatically learn the most relevant feature combinations, reducing the workload of artificial feature selection; it is an end-to-end perception and control system with strong versatility, can achieve direct control from raw input to output, without the intervention of intermediate processes, and can more directly reflect the relationship between the state of the environment and the action, thereby improving the efficiency and accuracy of learning; it can perform transfer learning in different tasks and environments to improve learning efficiency.

[0039] Furthermore, in deep reinforcement learning, a deep neural network is used to learn state representation, while reinforcement learning methods are used to learn strategies. The learning process can be described as follows: at each moment, the agent interacts with the environment to obtain a high-dimensional observation, and uses deep learning methods to perceive the observation to obtain a specific state feature representation. The value function of each action is evaluated based on the expected return, and the current state is mapped to the corresponding action through a certain strategy. The environment responds to this action and obtains the next observation. By continuously looping the above process, the optimal strategy to achieve the goal can be finally obtained. The above-mentioned agent in this embodiment may refer to a washing machine control system.

[0040] Furthermore, the core algorithms of deep reinforcement learning mainly include Deep Q-Network (DQN), Double DQN, Dueling DQN, Policy Gradient, etc. The deep Q network is a model that combines deep learning and Q learning. It uses a deep neural network to learn Q-values ​​to help agents make better decisions. In DQN, two neural networks with the same structure but different parameters are usually used, one with the latest parameters for predicting Q-value estimates, and the other with old parameters for predicting Q-value realities. This setting helps to disrupt correlations and improve the stability of learning. In this embodiment, the pre-trained deep reinforcement learning model is constructed using a DQN network for illustration.

[0041] In some examples, before performing feature analysis through a trained model, model training is required first. The training of the deep reinforcement learning model can be achieved by constructing a model training data set to obtain the above-mentioned pre-trained deep reinforcement learning model. Among them, the model training data set can be an offline data set and / or an online data set, and the data set can include washing parameters of detergents of various brands, state parameters of various types of clothing, washing effect data, user feedback data, etc., and data collection can be achieved based on different detergent supplier systems, network surveys, offline collection and other multi-source data collection methods. For DQN network training, the washing parameters of various detergents and the state parameters of various types of clothes in the model training data set are used as input vectors, and the detergent dosage is used as output. The training process includes: input layer: receiving the washing parameters of the detergent and the state parameters of the clothes, for example, the type and concentration of the detergent, the degree of stains, material, color, etc. of the clothes, and these parameters are converted into numerical form or vector representation. Hidden layer: composed of multiple neurons, used to extract the features of the input vector and learn the complex relationship between the detergent dosage and each input parameter. Output layer: output the Q value of the detergent dosage.

[0042] In this embodiment, after a pre-trained deep reinforcement learning model is obtained based on model training, in actual predictive use, after the washing characteristics of the detergent and the state characteristics of the clothes are acquired, the intelligent agent in the model can construct a feature vector based on the two, and the feature vector includes various characteristic parameters of the detergent and various characteristic parameters of the clothes. Combining the parameters of the detergent and the parameters of the clothes to construct a vector is conducive to the model combining the two for specific analysis, so as to obtain a more accurate amount of detergent.

[0043] In some optional examples, the water quality characteristics of the water used can also be obtained, and the water quality characteristics may include pH value, hardness, dissolved oxygen, heavy metal content, etc. If the water quality characteristics are obtained, the intelligent agent in the pre-trained deep reinforcement learning model can jointly construct a feature vector based on the washing characteristics of the detergent, the state characteristics of the laundry, and the water quality characteristics, which more completely combines the factors in multiple dimensions that affect the washing effect to participate in the analysis, which is conducive to more accurate expression of the analysis results.

[0044] Step S206: Perform feature analysis based on the feature vector using the pre-trained deep reinforcement learning model to determine a target vector, wherein each element in the target vector corresponds to a target estimated value of a detergent dispensing strategy.

[0045] In this embodiment, the intelligent agent can input the constructed feature vector into the DQN network in the pre-trained deep reinforcement learning model for model training, feature recognition, etc., and a target vector (Q value vector) will be output through the DQN network. The Q value vector includes multiple elements, each element corresponds to a target estimated value of a detergent dosage strategy or action, that is, a Q value estimate, and each Q value estimate corresponds to a different detergent dosage.

[0046] Step S208, based on the target estimated value of the detergent delivery strategy in each element of the target vector, the detergent delivery strategy corresponding to the maximum target estimated value is selected as the optimal detergent delivery strategy for controlling the delivery amount of the detergent delivery machine.

[0047] In some examples, based on the Q-value estimates in the output Q-value vector, for the discretized action space, the agent can select the action with the largest Q-value estimate as the optimal washing machine delivery strategy. The discretized action space means that the number of actions that the agent can perform is limited, and each action is selected from a predefined, discrete set. In this set, the agent cannot select actions outside the set, but can only select an action from the set to execute. For example, the amount of detergent delivered is discretized into 5 levels, namely, small amount, medium amount, standard amount, large amount, and excessive amount. The action space of the agent is a discrete set containing these 5 levels. When making decisions, the agent will select an action with the largest Q value (detergent delivery level) from the Q-value vector as the optimal strategy to execute.

[0048] In other examples, based on the estimates of each Q value in the output Q value vector, for the continuous action space, the agent can use a greedy strategy or an approximate optimal strategy to select the amount of detergent to be added. Among them, the continuous action space means that the agent can perform an infinite number of actions, and the action can be any value in the real number range. In this case, the agent cannot simply select an action from a discrete set, but needs to make a decision within the entire real number range. For example, the amount of detergent to be added is continuously adjustable from 0 to 100, and the action space of the agent corresponds to a continuous real number range. When making decisions, the agent can use a greedy strategy or an approximate optimal strategy to select the amount of detergent to be added, wherein the greedy strategy refers to the agent selecting the action that can maximize the Q value in the current state. In this way, the amount of detergent to be added is determined, and when selecting the approximate optimal strategy, more factors may be considered, such as possible future washing effects, the cost of the strategy, etc. to select the amount of the washing machine to be added.

[0049] In other examples, the optimal washing machine placement strategy is implemented to control the washing machine placement, and the execution results can be observed, such as washing effect, energy consumption, etc. These execution results can be used as feedback for subsequent training and optimization of the DQN network.

[0050] In this embodiment, by acquiring the washing characteristics of the detergent and the state characteristics of the laundry to construct a feature vector, the accuracy and dimension of the data parameters of the pre-trained deep reinforcement learning model can be improved, which is more conducive to the pre-trained deep reinforcement learning model to perform more accurate feature extraction and recognition in a complex environment through continuous trial and error and feedback learning, so as to select the optimal detergent delivery strategy, and realize more flexible and accurate automatic delivery control of the detergent based on the optimal detergent delivery strategy, so as to ensure that the laundry is fully cleaned while avoiding detergent waste, and is conducive to reducing pollution to the environment and reducing water waste. It can also meet consumers' needs for convenience and comfort by improving the intelligence level of washing machines.

[0051] In some optional embodiments, combined with Figure 2 As shown, in the above step S202: the step of obtaining the washing characteristics of the detergent includes:

[0052] S2021, collecting an image of the detergent, and preprocessing the image of the detergent using an optical character recognition model to obtain a preprocessed image;

[0053] S2022, performing text recognition and conversion on the pre-processed image to obtain detergent character information;

[0054] S2023, performing text analysis and information extraction based on the detergent character information to obtain washing characteristics of the detergent.

[0055] In this embodiment, a pre-trained optical character recognition model (Optical Character Recognition, OCR) can be used to extract washing features. OCR is a technical recognition technology that scans and recognizes text on paper documents and converts them into a digital text format that can be edited and processed by a computer. It specifically includes image preprocessing, text area detection, character segmentation, feature extraction and character recognition.

[0056] In some examples, an image of a detergent package or label can be acquired by using a camera or a scanner, and the acquired image can be preprocessed to obtain a preprocessed image, wherein the image preprocessing includes but is not limited to graying, binarization, denoising, and other operations to improve the accuracy of OCR recognition.

[0057] Furthermore, the trained OCR model can be used to locate and segment the text area in the preprocessed image. Specifically, the area in the image that may contain text can be found through image processing techniques such as edge detection and contour analysis. The detected text area is further segmented into individual characters based on the spacing and connectivity between characters, and the character image is converted into computer-readable detergent character information.

[0058] Furthermore, after obtaining the detergent character information, it is necessary to extract the desired useful information from the washing machine character information to obtain the washing characteristics of the detergent. Specifically, first, the OCR model will output the recognized detergent character information in the form of a text string, which contains all the text content in the image. According to the preset rules or algorithms, the text can be split into single words, phrases or sentences, and the keywords and contextual relationships therein can be identified. Among them, Natural Language Processing (NLP) word segmentation, part-of-speech tagging, named entity recognition and other operations can be used.

[0059] Furthermore, after the text is split, the desired washing features can be extracted from the text based on the text analysis, according to specific needs, such as extracting parameter information such as the composition and concentration of detergents or washing powder, using regular expressions, pattern matching or semantic understanding, etc. The washing features may be stored in key-value pairs, lists or structured data formats. In addition, the extracted washing features can be verified and cleaned to ensure their accuracy and consistency, for example, checking whether the values ​​are within a reasonable range, removing duplicate or invalid information, etc. Among them, washing features include but are not limited to ingredient names, the content or concentration of each ingredient, instructions for use, precautions, production date and expiration date, etc.

[0060] In this embodiment, by using OCR technology to identify and obtain the washing characteristics of the detergent, it can provide important input parameters for the subsequent amount of detergent; and OCR recognition can quickly and accurately convert documents into digital text, which greatly improves work efficiency. In addition, by using OCR technology to quickly identify various parameter information of detergents, it is more conducive to the pre-trained reinforcement learning model to predict the decontamination ability, so as to more accurately control the amount of detergent to be put.

[0061] In some optional embodiments, combined with Figure 3 As shown, in the above step S202: the step of obtaining the state characteristics of the laundry includes:

[0062] Collecting laundry images of laundry in multiple dimensions;

[0063] Performing feature recognition on the laundry images of each dimension through a pre-trained image recognition model to extract laundry features;

[0064] The stain features of the laundry are extracted according to the laundry features.

[0065] In this embodiment, the state feature includes a stain feature, and the stain includes but is not limited to oil stains, blood stains, mud stains, ink stains, and food residues, etc. Before the user puts the clothes into the washing machine, the camera installed on the washing machine can take images of the washing clothes from multiple positions such as the front and side of the user's clothes, wherein the camera automatically adjusts the focus and light to ensure the clarity and accuracy of the image.

[0066] Furthermore, the camera can transmit the collected laundry images to a pre-trained image recognition model for processing and analysis, wherein the pre-trained image recognition model can use image recognition algorithms, such as convolutional neural networks (CNN), image segmentation algorithms, etc., to compare and analyze information such as pixel values, textures, and colors in the image, quickly and accurately identify the size, color, and depth information of the laundry features, and select the stain features included therein based on the size, color, and depth information of the identified laundry features. Among them, the laundry features can include the color and depth information of the clothes themselves and the size, color, and depth information of the stains. The stain features are input into the subsequent pre-trained reinforcement learning model, combined with the acquired washing features of the detergent for analysis, and the optimal detergent dosage usage strategy is given.

[0067] In this embodiment, by collecting laundry images before the laundry is put in, extracting stain features from the collected laundry images, and performing feature analysis in combination with the washing characteristics of the stains and detergents, the pre-trained deep learning model can accurately match the most suitable detergent and amount to be put in according to information such as the type, area and severity of the stains, thereby making the washing effect more significant and being able to more effectively remove stains on clothes.

[0068] In some optional embodiments, in the above step S202: the step of obtaining the state characteristics of the laundry further includes:

[0069] When laundry is placed in the washing machine, the laundry weight characteristics, dirtiness parameters and fabric quality parameters of the laundry are respectively collected based on different sensors;

[0070] Performing data preprocessing on the dirtiness parameter and the fabric quality parameter, and performing primary feature extraction based on the preprocessed data;

[0071] Perform advanced feature extraction on the features obtained from the primary feature extraction to obtain advanced features of the degree of dirtiness and advanced features of fabric quality;

[0072] The features obtained by high-level feature extraction are subjected to feature selection and dimensionality reduction processing to obtain the dirtiness degree feature and the fabric quality feature respectively.

[0073] In some examples, combined Figure 4 As shown, before detecting the weight characteristics, dirtiness parameters and fabric quality parameters of the laundry, the terminal device can be controlled to be linked with the washing machine, and the washing mode of the washing machine can be started by controlling the terminal. The terminal device can be a smart electronic device such as a mobile phone.

[0074] In this embodiment, the state characteristics also include laundry weight characteristics, dirtiness characteristics, and fabric characteristics. Different sensor modules can be used to collect real-time data such as the weight, dirtiness, and fabric quality of the laundry. The sensors may include weight sensors, turbidity sensors, and material recognition sensors. The sensor sends the collected data to the data preprocessing module for data preprocessing. The data preprocessing module preprocesses and extracts features from the data collected by the sensor, and the processed data is input into the pre-trained deep reinforcement learning model.

[0075] It should be noted that sensors and cameras are complementary in function, and each is responsible for collecting different types of data. Sensors are mainly concerned with the physical properties of clothing, while cameras are more focused on the appearance characteristics of clothing. In addition, the above-mentioned camera-based collection of stain features, while the turbidity sensor here is used to detect different concentrations of dirt, for example: different types of gasoline have different concentrations. Therefore, the dirt degree parameters collected here are different from the stain features collected above.

[0076] Specifically, when the clothes are placed in the detergent, the weight sensor can directly collect the weight characteristics of the laundry. The collected weight characteristics of the laundry are specific weight data values, and no data preprocessing is required. However, the collected soil degree parameters and fabric parameters need to be specifically analyzed to be known, so data preprocessing operations can be performed on them, which may include data cleaning, normalization and standardization. Among them, data cleaning: For obvious abnormal values ​​in the soil degree parameters and fabric parameters, methods such as deletion, replacement or smoothing can be used for processing; for missing data, methods such as interpolation, mean filling, median filling or model-based prediction can be used for filling. Normalization and standardization: Data of different dimensions in the soil degree parameters and fabric parameters can be converted to the same scale for subsequent processing. For example, the fabric parameters and soil degree parameters can be converted into values ​​between 0 and 1, and the data can be converted into the form of standard normal distribution by calculating the mean and standard deviation of the values.

[0077] Furthermore, the preprocessed data can be subjected to primary feature extraction, advanced feature extraction, feature selection and dimensionality reduction processing, so as to extract the dirtiness and fabric features. For primary feature extraction: based on the dirtiness parameters collected by the turbidity sensor and preprocessed, the features representing the dirtiness can be extracted, including but not limited to the absolute value of the dirtiness, the distribution of the dirtiness, etc. According to the fabric parameters collected by the material recognition sensor and preprocessed, the features representing the fabric can be extracted, including the type, thickness, elasticity, etc. of the fabric, and the type, thickness, and elasticity can be converted into corresponding numerical values. For advanced feature extraction: by calculating the statistics of the extracted dirtiness and the primary features of the fabric, such as the mean, variance, skewness, kurtosis, etc., if the primary features of the dirtiness have the characteristics of periodicity or periodic changes, the frequency domain features can be extracted by Fourier transform and other methods. For advanced feature recognition of fabric, the texture features of the fabric can be extracted, including the gray level co-occurrence matrix and local binary pattern corresponding to the fabric. For feature selection and dimensionality reduction: correlation analysis, mutual information, recursive feature elimination and other methods can be used to select features that are more useful for the subsequent deep reinforcement learning module from the extracted high-level features of soil degree and cloth quality. If the feature dimension is too high, principal component analysis (PCA) and linear discriminant analysis (LDA) can be used to reduce the feature dimension. After feature selection and dimensionality reduction, the soil degree features and cloth quality features are finally output.

[0078] In this embodiment, by collecting the laundry state data of multiple dimensions of laundry weight characteristics, dirtiness characteristics and fabric characteristics to participate in the control of detergent input, and combining the washing characteristics of the detergent to jointly construct a feature vector as the input of the pre-trained deep reinforcement learning model, it is beneficial for the model to perform feature recognition and analysis according to the specific conditions of the detergent and the laundry, and accurately match the most suitable detergent and input amount according to the weight, dirtiness and fabric quality of the laundry, so that the washing effect is more significant and the stains on the clothes can be removed more effectively. In addition, through data preprocessing, primary feature extraction, advanced feature extraction, feature selection and dimensionality reduction processing, the accuracy of the features can be improved, which is convenient for the accurate recognition of the model.

[0079] In some optional embodiments, the above step S206 includes:

[0080] Receiving the feature vector through an input layer of the pre-trained deep reinforcement learning model, wherein the number of neurons in the input layer is consistent with the dimension of the feature vector;

[0081] The feature vector is nonlinearly transformed based on the hidden layer of the pre-trained deep reinforcement learning model and then output to the output layer, and the target vector is output based on the output layer.

[0082] In this embodiment, the pre-trained deep reinforcement learning model uses a DQN network as a policy network and a value network, including an input layer, a hidden layer, and an output layer, wherein the hidden layer includes at least one layer. Strategy updates and value evaluations can be performed by continuously receiving the acquired data and feedback of washing effect information. The input layer receives the feature vector of the state space; the hidden layer is used to extract the deep features of the state space; and the output layer outputs the Q value estimate of each action.

[0083] In some examples, in the DQN network, an experience replay buffer can be used to store the experience of the agent interacting with the environment. During the training process, a batch of experiences are randomly extracted from the experience replay buffer for parameter update to improve the stability and convergence of the training. At the same time, a target network with the same structure as the main network but fixed parameters is introduced to calculate the target Q value. The parameters of the target network are regularly updated to the parameters of the main network to reduce the fluctuation of the target value caused by the parameter update. In addition, the parameters of the DQN network need to be initialized, including the initialization learning rate, discount factor, experience replay size, network architecture, etc. In each training round, the agent selects an action based on the current state and strategy. After executing the action, observe the environmental feedback (i.e., the next state and reward). The current state, action, reward, and next state are stored in the experience replay buffer, and a batch of experiences are randomly extracted from the experience replay buffer for training the DQN network. The target Q value is calculated using the target network, and the parameters of the DQN network are optimized using the back propagation algorithm. The above process of initialization to back propagation optimization of network parameters is repeated until the DQN network converges or reaches a preset training round.

[0084] In this embodiment, the above-mentioned feature vector may include the type code of detergent, the area ratio of oil stains, the weight category of clothes, the type code of cloth, the degree of oil stains, the water hardness level, the pH value and other features. When input into the DQN network, it is received through the input layer, where the number of neurons in the input layer matches the dimension of the feature vector. For example, the feature vector contains 5 features, and the input layer corresponds to 5 neurons. Then, the feature vector will pass through the forward propagation process of the network, through the nonlinear transformation of the hidden layer, and finally reach the output layer, and the output layer will output the target vector, which includes the Q value estimate of each possible action. Among them, the output Q value estimate corresponds to the detergent delivery strategy, which includes but is not limited to the amount of detergent delivered, the delivery of water, the selection of washing programs, the washing time and the number of rinses, etc. For example, when the washing machine detects that the weight of the clothes is heavy and the degree of dirtiness is high, the deep reinforcement learning module will output a larger amount of detergent delivered; and when the weight of the clothes is light and the degree of dirtiness is low, a smaller amount of detergent delivered will be output.

[0085] In this embodiment, the DQN network has strong learning ability and adaptability. It can learn and adjust according to the characteristics of different detergents and laundry, and can accurately and dynamically adjust the amount of detergent to be added, ultimately achieving intelligent laundry under the optimal detergent delivery strategy, providing users with a better experience, not only ensuring that the clothes are fully cleaned, but also ensuring the best washing effect in all situations. In addition, by accurately controlling the amount of detergent input, detergent waste can be avoided. The application of the DQN network makes the control of the amount of detergent input more intelligent. Users do not need to manually adjust the amount of detergent. They only need to put the clothes into the washing machine and select the appropriate washing program. The system can automatically calculate and deliver the amount of detergent input, which is more flexible.

[0086] In some optional embodiments, after the hidden layer based on the pre-trained deep reinforcement learning model performs nonlinear transformation on the feature vector and outputs it to the output layer, and after the target vector is output based on the output layer, the method further includes:

[0087] Based on the online network in the pre-trained deep reinforcement learning model, predicting an estimated value of the current state according to the current state and the detergent placement strategy, and based on the target network in the pre-trained deep reinforcement learning model, predicting a target estimated value of the next state;

[0088] Calculating a loss based on an estimate of the current state and the target estimate;

[0089] Based on the loss, the network weights in the pre-trained deep reinforcement learning model are updated through back propagation.

[0090] During the training process, the DQN network will calculate the loss based on the output Q value estimate and the actual reward value, and update the network weights through the back-propagation algorithm. The training goal of the DQN network is to make the target estimate output by the network closer to the estimated value of the current state. In order to reduce the error between the two, DQN can use a loss function to measure the error between the two. Among them, the loss function can include mean square error or other error measurement methods.

[0091] In some examples, the above DQN network can use two neural networks: one is an online network (EvalNet) for selecting actions and predicting Q values ​​(estimates of the current state); the other is a target network (Target Net) for calculating the target Q value (target estimate). The weights of the target network are periodically copied from the weights of the online network to increase the stability of learning. The target Q value is calculated as follows:

[0092] Use the online network to predict the Q value of the current state S(t) and select an action A according to the ε-greedy strategy. Execute this action and observe the next state S(t+1) and the reward R obtained. Use the target network to calculate the maximum predicted Q value of the next state S(t+1), that is, max Q(S(t+1), a'), where a' is all possible actions under the next state S(t+1). Combine the reward R of the current state to calculate the target Q value, that is, the target Q value y=R+γ*max Q(S(t+1), a'), where γ is the discount factor, indicating the degree of discount of future rewards. Then, calculate the error (loss) between the Q value Q(S(t), A) predicted by the online network and the target Q value y, that is, L=(Q(S(t), A)-y) 2 .

[0093] Furthermore, after obtaining the loss value, the DQN network uses the back propagation algorithm to update the network weights of the online network to minimize the error L. This process is repeated until the network converges to a stable strategy, thereby achieving model optimization, which is conducive to more accurate feature training and recognition of the model. Among them, the method of updating the network weights includes but is not limited to: calculating the gradient of the loss function with respect to the network output; using the chain rule to calculate the gradient of the loss function with respect to the network weights; using the gradient descent algorithm to update the network weights.

[0094] In this embodiment, by calculating the error of the estimated outputs of the two networks in the DQN network and updating the network weights of the model based on the error back propagation, the weight parameters of the neural network can be optimized to make more accurate predictions, which helps the agent make more informed decisions when selecting actions and select the optimal washing machine placement strategy. In addition, in the DQN network, the weights of the target network are regular copies of the online network weights, which helps to increase the stability of learning. When updating the weights of the online network through back propagation, since the target network remains unchanged, the problem of training instability caused by frequent changes in network weights can be avoided.

[0095] In some optional embodiments, after the above step S208, the method further includes:

[0096] Acquiring execution result data of detergent input control based on the optimal detergent input strategy;

[0097] A new feature vector is constructed based on the execution result data, the washing characteristics of the detergent and the state characteristics of the laundry to optimize the pre-trained deep reinforcement learning model.

[0098] In this embodiment, combined with Figure 4 As shown in the figure, after each washing of clothes according to the selected optimal detergent delivery strategy, the execution data, that is, the execution results, including cleanliness, energy consumption, detergent consumption, etc., can be fed back. Then, a new feature vector is constructed by combining the execution result data, the washing characteristics of the detergent and the state characteristics of the laundry, and re-entered into the pre-trained deep reinforcement learning model for optimization, which is conducive to better meeting user needs and thus improving user experience.

[0099] In some embodiments, the reward function can also be adjusted according to actual application requirements to guide the agent to learn a more desired detergent delivery strategy. The reward function is a function used to evaluate the reward or penalty obtained by the agent after selecting an action in a given state, for example, evaluating the execution result obtained by adding 100 ml of detergent in the current state. The actual application requirements may include washing effect, detergent consumption, energy consumption, washing time, user satisfaction, etc.

[0100] Furthermore, for washing effect: if the application requirements emphasize washing effect, reward items directly related to washing effect can be added to the reward function. For example, when the cleanliness of the clothes after washing reaches a certain threshold, a higher reward is given; otherwise, a lower reward or penalty is given. For detergent consumption: if the application requirements focus on detergent consumption, reward items related to detergent consumption can be added to the reward function. For example, when the detergent consumption is lower than a certain threshold, a reward is given; otherwise, a penalty is given. For energy consumption: for energy-saving washing machines, energy consumption factors can be considered in the reward function. For example, when the energy consumption during the washing process is low, a reward is given; otherwise, a penalty is given. For washing time: if the application requirements have requirements for washing time, reward items related to washing time can be added to the reward function. For example, when the washing time is short and the washing effect is good, a reward is given. For user satisfaction: in some user interaction scenarios, user satisfaction can be considered as part of the reward function. For example, the washing effect is evaluated through user feedback or ratings, and rewards or penalties are given based on the evaluation results.

[0101] In other examples, a multi-objective reward function can be introduced to balance multiple performance indicators such as washing effect and detergent consumption. Among them, the multi-objective mainly includes two or more of the above, and not only includes the above practical application needs. For example, for a smart washing machine, if it is necessary to select a detergent delivery strategy based on the degree and type of stains on the clothes, the ultimate goal is to ensure good washing results while minimizing detergent consumption.

[0102] In this regard, the process of introducing a multi-objective reward function to balance the performance indicators of multiple aspects such as washing effect and detergent consumption includes: defining a multi-objective reward function: including a washing effect reward item (evaluated according to the cleanliness of the clothes after washing, the higher the cleanliness, the higher the reward) and a detergent consumption penalty item, wherein the washing effect reward item is evaluated according to the cleanliness of the clothes after washing, the higher the cleanliness, the higher the reward; the detergent consumption penalty item is evaluated according to the consumption of detergent, the greater the consumption, the higher the penalty. In addition, it also includes: setting appropriate weights for the washing effect reward item and the detergent consumption penalty item according to actual needs. For example, if you pay more attention to the washing effect, you can set a higher weight for the washing effect reward item; if you pay more attention to detergent consumption, you can set a higher weight for the detergent consumption penalty item. In addition, calculate the total reward: according to the detergent delivery strategy selected by the agent, calculate the values ​​of the washing effect reward item and the detergent consumption penalty item, and calculate the total reward according to the set weight. Finally, select the optimal detergent delivery strategy: the agent selects the optimal detergent delivery strategy according to the size of the total reward, wherein the strategy with the highest total reward is the optimal detergent delivery strategy.

[0103] According to another aspect of the embodiment of the present application, Figure 5 As shown, corresponding to the detergent control method based on deep learning in the above embodiment, this embodiment provides a detergent control device based on deep learning, and the device includes:

[0104] The data acquisition module 501 is used to acquire the washing characteristics of the detergent and the state characteristics of the laundry;

[0105] A vector construction module 503, configured to construct a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model;

[0106] A feature extraction module 505 is used to perform feature analysis based on the feature vector through the pre-trained deep reinforcement learning model to determine a target vector, wherein each element in the target vector corresponds to a target estimated value of a detergent dispensing strategy;

[0107] The control module 507 is used to select the detergent dispensing strategy corresponding to the maximum target estimated value as the optimal detergent dispensing strategy for controlling the detergent dispensing amount of the washing machine based on the target estimated value of the detergent dispensing strategy in each element of the target vector.

[0108] It should be noted that, in this embodiment, the data acquisition module 501 can be used to execute step S202 in the embodiment of the present application, the vector construction module 503 in this embodiment can be used to execute step S204 in the embodiment of the present application, the feature extraction module 505 in this embodiment can be used to execute step S206 in the embodiment of the present application, and the control module 507 in this embodiment can be used to execute step S208 in the embodiment of the present application.

[0109] Optionally, the data acquisition module 501 includes: a first image acquisition submodule, used to acquire an image of the detergent, and preprocess the image of the detergent through an optical character recognition model to obtain a preprocessed image; a graphic and text recognition submodule, used to perform text recognition and conversion on the preprocessed image to obtain detergent character information; an information extraction submodule, used to perform text analysis and information extraction based on the detergent character information to obtain the washing characteristics of the detergent.

[0110] Optionally, the state feature includes a stain feature, and the data acquisition module 501 further includes: a second image acquisition submodule, used to acquire laundry images of laundry in multiple dimensions; a feature recognition submodule, used to perform feature recognition on the laundry images of each dimension through a pre-trained image recognition model to extract laundry features; and a first feature extraction submodule, used to extract the stain features of the laundry based on the laundry features.

[0111] Optionally, the state characteristics also include laundry weight characteristics, soil degree characteristics and fabric quality characteristics, and the data acquisition module 501 also includes: a detection data acquisition submodule, which is used to collect the laundry weight characteristics, soil degree parameters and fabric quality parameters of the laundry based on different sensors when the laundry is placed in the washing machine; a second feature extraction submodule, which is used to perform data preprocessing on the soil degree parameters and fabric quality parameters, and perform primary feature extraction based on the preprocessed data; a third feature extraction submodule, which is used to perform high-level feature extraction on the features obtained by the primary feature extraction to obtain high-level soil degree features and high-level fabric quality features; a feature selection and dimensionality reduction submodule, which is used to perform feature selection and dimensionality reduction on the features obtained by the high-level feature extraction to obtain the soil degree features and the fabric quality features, respectively.

[0112] Optionally, the feature extraction module 505 includes: a feature receiving submodule, used to receive the feature vector through the input layer of the pre-trained deep reinforcement learning model, wherein the number of neurons in the input layer is consistent with the dimension of the feature vector; a feature transformation submodule, used to perform a nonlinear transformation on the feature vector based on the hidden layer of the pre-trained deep reinforcement learning model and output it to the output layer, and output the target vector based on the output layer.

[0113] Optionally, the device also includes: a first calculation module, used to predict an estimated value of the current state based on the online network in the pre-trained deep reinforcement learning model according to the current state and the detergent dosage strategy, and to predict a target estimated value of the next state based on the target network in the pre-trained deep reinforcement learning model; a second calculation module, used to calculate the loss based on the estimated value of the current state and the target estimated value; and a model updating module, used to update the network weights in the pre-trained deep reinforcement learning model through back propagation based on the loss.

[0114] Optionally, the device also includes: an execution result acquisition module, used to obtain execution result data of detergent dosage control based on the optimal detergent dosage strategy; a model optimization module, used to construct a new feature vector based on the execution result data, the washing characteristics of the detergent and the state characteristics of the laundry to optimize the pre-trained deep reinforcement learning model.

[0115] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules as part of the device can be run in Figure 1 In the hardware environment shown, it can be implemented by software or by hardware.

[0116] It should be noted here that the suffixes such as module, component, unit, sub-module, and sub-unit used to represent elements described in the above device are only for the convenience of description of this application and have no specific meaning in themselves. Therefore, they can be used in combination.

[0117] According to another aspect of the embodiments of the present application, the present application provides a washing machine, which controls the automatic addition of detergent according to the above-mentioned deep learning-based detergent control method. The washing machine is an intelligent washing machine and can implement the various embodiments of the deep learning-based detergent control method and achieve corresponding technical effects, which will not be repeated here.

[0118] According to another aspect of the embodiment of the present application, the present application provides a computer device, such as Figure 6 As shown, it includes a memory 601, a processor 603, a communication interface 605 and a communication bus 607. The memory 601 stores a computer program that can be run on the processor 603. The memory 601 and the processor 603 communicate through the communication interface 605 and the communication bus 607. When the processor 803 executes the computer program, the steps of the detergent control method based on deep learning are implemented.

[0119] The memory and processor in the above-mentioned computer device communicate through a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0120] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0121] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0122] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of the detergent control method based on deep learning in any of the above embodiments.

[0123] Optionally, in an embodiment of the present application, a computer-readable medium is configured to store program code for the processor to execute the steps of the deep learning-based detergent control method described in the above embodiment.

[0124] Optionally, the specific examples in this embodiment can refer to the examples described in the above embodiments, and this embodiment will not be repeated here. And when the embodiments of this application are specifically implemented, they can refer to the above embodiments and have corresponding technical effects.

[0125] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0126] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0127] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0131] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0132] It should be noted that, in this article, relational terms such as first, second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms include, include or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0133] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A detergent control method based on deep learning, characterized in that: The method comprises: Acquiring washing characteristics of the detergent and acquiring state characteristics of the laundry; Constructing a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model; Performing feature analysis based on the feature vector using the pre-trained deep reinforcement learning model to determine a target vector, wherein each element in the target vector corresponds to a target estimated value of a detergent placement strategy; Based on the target estimated value of the detergent delivery strategy in each element of the target vector, the detergent delivery strategy corresponding to the maximum target estimated value is selected as the optimal washing machine delivery strategy to control the washing machine delivery amount.

2. The detergent control method based on deep learning according to claim 1, characterized in that: The step of obtaining the washing characteristics of the detergent comprises: Collecting an image of detergent, and preprocessing the image of detergent by using an optical character recognition model to obtain a preprocessed image; Performing text recognition and conversion on the pre-processed image to obtain detergent character information; Text parsing and information extraction are performed based on the detergent character information to obtain the washing characteristics of the detergent.

3. The detergent control method based on deep learning according to claim 1, characterized in that: The state characteristics include stain characteristics, and the step of obtaining the state characteristics of the laundry includes: Collecting laundry images of laundry in multiple dimensions; Performing feature recognition on the laundry images of each dimension through a pre-trained image recognition model to extract laundry features; The stain features of the laundry are extracted according to the laundry features.

4. The detergent control method based on deep learning according to claim 3, characterized in that: The state characteristics also include laundry weight characteristics, dirtiness characteristics, and fabric characteristics. The step of obtaining the laundry state characteristics also includes: When laundry is placed in the washing machine, the laundry weight characteristics, dirtiness parameters and fabric quality parameters of the laundry are respectively collected based on different sensors; Performing data preprocessing on the dirtiness parameter and the fabric quality parameter, and performing primary feature extraction based on the preprocessed data; Perform advanced feature extraction on the features obtained from the primary feature extraction to obtain advanced features of the degree of dirtiness and advanced features of fabric quality; The features obtained by high-level feature extraction are subjected to feature selection and dimensionality reduction processing to obtain the dirtiness degree feature and the fabric quality feature respectively.

5. The detergent control method based on deep learning according to claim 1, characterized in that: The pre-trained deep reinforcement learning model performs feature analysis according to the feature vector to determine the target vector, including: Receiving the feature vector through an input layer of the pre-trained deep reinforcement learning model, wherein the number of neurons in the input layer is consistent with the dimension of the feature vector; The feature vector is nonlinearly transformed based on the hidden layer of the pre-trained deep reinforcement learning model and then output to the output layer, and the target vector is output based on the output layer.

6. The detergent control method based on deep learning according to claim 5, characterized in that: After the hidden layer based on the pre-trained deep reinforcement learning model performs nonlinear transformation on the feature vector and outputs it to the output layer, and after the target vector is output based on the output layer, the method further includes: Based on the online network in the pre-trained deep reinforcement learning model, predicting an estimated value of the current state according to the current state and the detergent placement strategy, and based on the target network in the pre-trained deep reinforcement learning model, predicting a target estimated value of the next state; Calculating a loss based on an estimate of the current state and the target estimate; Based on the loss, the network weights in the pre-trained deep reinforcement learning model are updated through back propagation.

7. The detergent control method based on deep learning according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquiring execution result data of detergent input control based on the optimal detergent input strategy; A new feature vector is constructed based on the execution result data, the washing characteristics of the detergent and the state characteristics of the laundry to optimize the pre-trained deep reinforcement learning model.

8. A detergent control device based on deep learning, characterized in that: The device comprises: A data acquisition module, used to acquire the washing characteristics of the detergent and the state characteristics of the laundry; A vector construction module, used to construct a feature vector based on the washing characteristics of the detergent and the state characteristics of the laundry through a pre-trained deep reinforcement learning model; A feature extraction module, configured to perform feature analysis based on the feature vector through the pre-trained deep reinforcement learning model to determine a target vector, wherein each element in the target vector corresponds to a target estimated value of a detergent placement strategy; The control module is used to select the detergent delivery strategy corresponding to the maximum target estimated value as the optimal washing machine delivery strategy to control the washing machine delivery amount based on the target estimated value of the detergent delivery strategy in each element of the target vector.

9. A washing machine, characterized in that: The washing machine controls automatic addition of detergent according to the detergent control method based on deep learning as described in any one of claims 1 to 7 above.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the steps of the detergent control method based on deep learning as described in any one of claims 1 to 7.

Images