Control method and device of water dispenser and water dispenser

Abstract

The present application relates to the field of household appliances, and provides a control method and device of a water dispenser and the water dispenser, the method comprising: acquiring a target cup image, the target cup image comprising a target brewing object; performing instance segmentation and cutting processing on the target cup image to obtain a target image; determining target category information of the target brewing object based on the target image; and outputting a control instruction for controlling the water temperature of the water dispenser based on the target category information. The method makes the identification of the target image pay more attention to the details of the target brewing object through instance segmentation and cutting processing, improves the identification accuracy and identification rate of the target brewing object, and outputs the control instruction to adjust the water temperature of the water dispenser after identifying the target category information of the target brewing object, thereby improving the brewing quality of the target brewing object and further improving the drinking experience of the user.

Description

Technical Field

[0001] This invention relates to the field of household appliance technology, and in particular to a control method, device, and water dispenser for a water dispenser. Background Technology

[0002] The variety and quantity of brewed beverages such as tea, coffee, and milk powder are increasingly abundant, effectively meeting diverse consumer needs. Currently, users obtain water from water dispensers to brew these products, but the fixed water temperature from these dispensers prevents most products from being brewed with water at the appropriate temperature, affecting brewing quality and the user's drinking experience. Summary of the Invention

[0003] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a control method for a water dispenser, which adjusts the water temperature to improve the quality of beverage preparation and thus enhance the user's drinking experience.

[0004] A control method for a water dispenser according to a first aspect of the present invention includes:

[0005] Acquire a target cup image, the target cup image including a target brewing object;

[0006] The target cup image is obtained by performing instance segmentation and cropping.

[0007] Based on the target image, determine the target category information of the target brewing object;

[0008] Based on the target category information, a control command is output to control the water temperature of the water dispenser.

[0009] According to the control method of the water dispenser of the present invention, the position of the target brewing object in the target cup image is determined by instance segmentation, and then the target cup image is cropped to remove interference from the non-target brewing object parts of the target cup image. This makes the recognition of the cropped target image more focused on the details of the target brewing object, improving the recognition accuracy and recognition speed of the target brewing object. After the target category information of the target brewing object is identified, a control command is output to adjust the water temperature of the water dispenser, improve the brewing quality of the target brewing object, and thus improve the user's drinking experience.

[0010] According to one embodiment of the present invention, determining the target category information of the target brewing object based on the target image includes:

[0011] The target image is convolved to obtain a first feature vector;

[0012] The first feature vector is pooled to obtain multiple second feature vectors of different dimensions and sizes;

[0013] The multiple second feature vectors are classified to obtain the target category information.

[0014] According to an embodiment of the present invention, the pooling process performed on the first feature vector to obtain multiple second feature vectors of different dimensions includes:

[0015] The first feature vector is subjected to global max pooling and global average pooling respectively to obtain the third feature vector and the fourth feature vector.

[0016] Based on the third feature vector and the fourth feature vector, the max pooling feature vector set and the average pooling feature vector set are obtained;

[0017] The max pooling feature vector group includes multiple second feature vectors of different dimensions and sizes, and the average pooling feature vector group includes multiple second feature vectors of different dimensions and sizes.

[0018] The classification process of the plurality of second feature vectors to obtain the target category information includes:

[0019] The target category information is obtained by classifying the max-pooling feature vector group and the average-pooling feature vector group.

[0020] According to an embodiment of the present invention, after obtaining the max-pooling feature vector set and the average-pooling feature vector set based on the third feature vector and the fourth feature vector, the method further includes:

[0021] Weights are extracted from the third feature vector to obtain multiple first weight values;

[0022] Weights are extracted from the fourth feature vector to obtain multiple second weight values;

[0023] The classification process of the max-pooling feature vector group and the average-pooling feature vector group to obtain the target category information includes:

[0024] The plurality of first weight values ​​are assigned to the plurality of second feature vectors corresponding to the max-pooling feature vector group;

[0025] The plurality of second weight values ​​are assigned to the plurality of second feature vectors corresponding to the average pooling feature vector group;

[0026] The weighted max-pooling feature vector group and the average-pooling feature vector group are classified to obtain the target category information.

[0027] According to an embodiment of the present invention, the step of performing convolution processing on the target image to obtain a first feature vector includes:

[0028] The target image is subjected to multi-layer convolution processing, and the image features after each layer of convolution processing are downsampled;

[0029] The image features after downsampling at each layer are fused with the image features obtained by the convolutional processing at the next layer.

[0030] The image features obtained from the last convolutional layer are used as the first feature vector.

[0031] According to an embodiment of the present invention, the step of performing convolution processing on the target image to obtain a first feature vector includes:

[0032] The target image is subjected to multi-layer convolution processing, and the image features after each layer of convolution processing are upsampled.

[0033] The image features upsampled at each layer are fused with the image features obtained from the convolutional processing of the previous layer.

[0034] The image features after feature fusion are convolved, and the image features after each convolutional layer are downsampled.

[0035] The image features after downsampling at each layer are fused with the image features obtained by the convolutional processing at the next layer.

[0036] The image features obtained from the last convolutional layer are used as the first feature vector.

[0037] According to one embodiment of the present invention, determining the target category information of the target brewing object based on the target image includes:

[0038] The target image is input into the target classification model to obtain the target category information output by the target classification model;

[0039] The target classification model is trained using sample images as samples and pre-determined category information corresponding to the sample brewing objects in the sample images as sample labels.

[0040] A control device for a water dispenser according to a second aspect embodiment of the present invention includes:

[0041] The acquisition module is used to acquire a target cup image, wherein the target cup image includes a target brewing object;

[0042] The first processing module is used to perform instance segmentation and cropping processing on the target cup image to obtain the target image;

[0043] The second processing module is used to determine the target category information of the target brewing object based on the target image;

[0044] The third processing module is used to output control commands for controlling the water temperature of the water dispenser based on the target category information.

[0045] A water dispenser according to a third aspect embodiment of the present invention includes:

[0046] A camera, used to capture images of the target cup;

[0047] As described above, the control device for the water dispenser is electrically connected to the camera.

[0048] An electronic device according to a fourth aspect of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the control method for the water dispenser as described above.

[0049] According to a fifth aspect of the present invention, a non-transitory computer-readable storage medium is provided thereon storing a computer program that, when executed by a processor, implements the steps of the control method for the water dispenser as described above.

[0050] According to a sixth aspect of the present invention, a computer program product includes a computer program that, when executed by a processor, implements the steps of the control method for a water dispenser as described above.

[0051] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0052] The location of the target brewing object in the target cup image is determined by instance segmentation. Then, the target cup image is cropped to remove interference from non-target brewing object parts. This makes the recognition of the cropped target image more focused on the details of the target brewing object, improving the recognition accuracy and speed. After the target category information of the target brewing object is identified, control commands are output to adjust the water temperature of the water dispenser, improving the brewing quality of the target brewing object and thus enhancing the user's drinking experience.

[0053] Furthermore, by fusing different types of pooling operations, the pooling information of the target image is fused, enriching the overall shape features of the target brewing object, further improving the detail recognition accuracy of the target image, and making the target category information corresponding to the identified target brewing object more accurate.

[0054] Furthermore, feature fusion can enrich the underlying texture details of the target brewing object, improve the detail recognition accuracy of the target image, and make the target category information corresponding to the identified target brewing object more accurate.

[0055] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the control method for a water dispenser provided in an embodiment of the present invention;

[0058] Figure 2 This is one of the schematic diagrams of the target image processing flow provided in the embodiments of the present invention;

[0059] Figure 3 This is a second schematic diagram of the target image processing flow provided in an embodiment of the present invention;

[0060] Figure 4 This is the third schematic diagram of the target image processing flow provided in the embodiments of the present invention;

[0061] Figure 5 This is the fourth schematic diagram of the target image processing flow provided in the embodiments of the present invention;

[0062] Figure 6 This is the fifth schematic diagram of the target image processing flow provided in the embodiments of the present invention;

[0063] Figure 7 This is the sixth schematic diagram of the target image processing flow provided in the embodiments of the present invention;

[0064] Figure 8 This is a schematic diagram of the control device for a water dispenser provided in an embodiment of the present invention;

[0065] Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0066] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0067] In the description of the embodiments of the present invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of the present invention. In addition, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0068] In the description of the embodiments of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention based on the specific circumstances.

[0069] In embodiments of the present invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0070] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0071] The following is combined Figures 1 to 7The present invention describes a control method for a water dispenser that, by identifying the category information of the object being brewed, adjusts the water temperature of the dispenser to improve the quality of beverage brewing and thus enhance the user's drinking experience.

[0072] The execution entity of the water dispenser control method in this embodiment of the invention can be the controller of the water dispenser, or the cloud, or an edge server.

[0073] In this embodiment of the invention, the water dispenser can be an under-sink water purifier, a wall-mounted water dispenser, a countertop water dispenser, a standing water dispenser, a countertop water purifier, a standing water purifier, a sparkling water dispenser, or a commercial water purifier, etc.

[0074] like Figure 1 As shown, the control method of the water dispenser in this embodiment of the invention includes steps 110 to 140.

[0075] Step 110: Obtain the image of the target cup.

[0076] In this step, a camera can be installed at the water dispenser's water outlet. When the user takes water from the cup near the water outlet, the camera captures an image of the cup used by the user as the target cup image.

[0077] The target cup image includes the cup used by the user and the target brewing object placed in the cup.

[0078] In this embodiment, the target brewing object refers to an item that can be used to brew water for drinking. The target brewing object can be tea leaves, tea bags, coffee, milk powder, or other items that can be brewed and drunk.

[0079] Step 120: Perform instance segmentation and cropping on the target cup image to obtain the target image.

[0080] Instance segmentation can separate objects from the background, detect objects in an image, and segment the image into different pixel regions according to different objects.

[0081] In practice, instance segmentation can also perform preliminary classification of the category of the object corresponding to each pixel region. For example, by performing instance segmentation on the target cup image, it can be determined that the target brewing object in the cup used by the user is tea or coffee.

[0082] In this embodiment, the target cup image is obtained by cropping the pixel region corresponding to the target brewing object obtained by instance segmentation.

[0083] Understandably, cropping can remove interference from the background outside the user's cup and the target brewing object in the target cup image, effectively improving the recognition accuracy and speed of the target brewing object in the target image.

[0084] In practice, the target image can be obtained by making tangents at different boundaries of the pixel area corresponding to the target brewing object, using the intersection of the four tangents to form an external rectangle, and then cropping the target cup image.

[0085] After cropping the target cup image to obtain the target image, the background outside the pixel area corresponding to the target brewing object in the target image can be completely blackened. That is, the pixel value corresponding to the point outside the pixel area corresponding to the target brewing object is set to 0, which reduces the amount of computation for subsequent image recognition and further improves the recognition accuracy and recognition speed.

[0086] In practice, the boundary of the pixel area corresponding to the target brewing object can be reduced by 2 to 5 pixels, and the value of the boundary 2 to 5 pixels can be set to 0 to reduce the impact of sudden changes in pixel area attributes on image recognition.

[0087] Step 130: Based on the target image, determine the target category information of the target brewing object.

[0088] In this step, image recognition can be used to identify details in the target image, extract feature information from the target image, and determine the target category information corresponding to the target brewing object in the target image.

[0089] Understandably, after the two image preprocessing processes of instance segmentation and cropping, the pixel ratio of the target brewing object in the target image is higher than that of the background, which greatly improves the recognition accuracy and recognition speed of the image recognition.

[0090] In this embodiment, the target category information corresponding to the target brewing object can be determined by acquiring the texture features of the target brewing object in the target image. The texture features of the target brewing object reflect the surface structure organization and arrangement attributes of the target brewing object.

[0091] For example, tea leaves have different textures on their surface. By obtaining the texture features corresponding to the textures on the surface of the target brewing object, the target category information of the tea can be determined.

[0092] In this embodiment, the target category information corresponding to the target brewing object can be determined by acquiring the shape features of the target brewing object in the target image. The shape features of the target brewing object reflect the surface structure organization and arrangement attributes of the target brewing object.

[0093] For example, tea leaves take on different shapes depending on the degree of roasting. By obtaining the shape features corresponding to the surface texture of the target brewing object, the target category information of the tea can be determined.

[0094] For example, different types of coffee powder have different grinds, resulting in different particle sizes. By obtaining the shape features corresponding to the surface texture of the target brewing object, the target category information of the coffee powder can be determined.

[0095] Target category information refers to the type of target brewing object. Compared to instance segmentation, which identifies the target brewing object in the cup as tea or coffee, the target category information determined by the target image is more specific and detailed.

[0096] For example, instance segmentation can identify the target brewing object in the cup as tea leaves, while image recognition can determine that the target brewing object in the target image is tea leaves such as green tea, black tea, white tea, yellow tea, oolong tea, or dark tea.

[0097] For example, instance segmentation can identify the target brewing object in the cup as coffee granules, while image recognition can determine whether the target brewing object in the target image is 3-in-1 coffee granules or black coffee granules.

[0098] Step 140: Based on the target category information, output the control command for controlling the water temperature of the water dispenser.

[0099] Based on the target category information of the target brewing object, the appropriate water temperature for brewing the target brewing object can be obtained, and then a control command can be output to adjust the water temperature of the water dispenser to the appropriate brewing temperature.

[0100] For example, image recognition can identify the target brewing object in the image as black tea. Based on the experience of tea experts, the appropriate water temperature for brewing black tea is determined to be 100℃, which is conducive to the release of aroma substances in black tea, improves the quality of the brewed tea, and thus enhances the user's drinking experience.

[0101] In this step, after outputting the control command to adjust the water temperature of the water dispenser according to the target category information, the user can control the water dispenser to dispense water at a suitable brewing temperature by clicking the physical button or screen on the water dispenser again.

[0102] In this step, after outputting the control command to adjust the water temperature of the water dispenser, the water dispenser can automatically dispense water at a suitable brewing temperature without user operation.

[0103] Among related technologies, there are techniques that use tea images to determine the type of tea and control the tea brewing process. These technologies typically do not crop the images and only identify the color of the tea leaves. When there are overlapping objects that severely obscure the image, it can easily lead to misjudgment of the type of tea.

[0104] This invention cropped the target cup image using the result of instance segmentation, removing interference from non-target brewing objects, allowing the target image recognition to focus more on the details of the target brewing object, thus improving the recognition accuracy and speed of the target brewing object.

[0105] According to the control method of the water dispenser provided by the present invention, the position of the target brewing object in the target cup image is determined by instance segmentation, and then the target cup image is cropped to remove interference from the non-target brewing object parts in the target cup image. This makes the recognition of the cropped target image more focused on the details of the target brewing object, improving the recognition accuracy and recognition speed of the target brewing object. After the target category information of the target brewing object is identified, a control command is output to adjust the water temperature of the water dispenser, improve the brewing quality of the target brewing object, and thus improve the user's drinking experience.

[0106] In some embodiments, step 130 may perform convolution processing, pooling processing, and classification processing on the target image to obtain target category information corresponding to the target brewing object.

[0107] In practice, convolutional neural networks with image classification capabilities can be used to perform convolution, pooling, and classification processing on target images.

[0108] Convolutional neural networks automatically learn the features of an image at various levels through convolution and pooling operations. They first learn local details such as color, brightness, edges, corners, or lines, then learn structures such as texture and geometric shapes, and finally form the concept of the overall object.

[0109] The target image is convolved to reduce its dimensionality and extract its features, resulting in an abstract representation of the target image at different scales. A first feature vector is then output.

[0110] The cropped target image is used as input and fed into the convolutional layer of a convolutional neural network or a backbone network for feature extraction. After layers of convolutional pooling operations, features of dimension CxWxH are obtained, where C refers to the output channel, W refers to the width of the feature, and H refers to the height of the feature.

[0111] In actual execution, C can be 2048, and the target image is convolved to obtain a first feature vector with a dimension of 2048x1x1.

[0112] Through convolution, the input target image undergoes dimensionality reduction and feature extraction. However, the resulting first feature vector has a high dimension. Pooling can be used to replace a region of the image with a single value. Using the maximum value is called max pooling, while using the mean value is called average pooling. Pooling reduces the dimension of the first feature vector, which helps improve the speed of subsequent calculations.

[0113] In this embodiment, the pooling process performed on the first feature vector can be global average pooling (Avg pool) or global max pooling (maxpool).

[0114] In practice, pooling can reduce the dimensionality of the first feature vector, and second feature vectors of different dimensions can be obtained based on the first feature vector.

[0115] For example, if the first feature vector is a vector of size 2048x1x1, the pooling operation reduces the dimension of the first feature vector by truncating 3 / 4, 1 / 2, and 1 / 4 of the first feature vector, respectively, and outputs a second feature vector of size 1536, a second feature vector of size 1024, and a second feature vector of size 512.

[0116] Multiple second feature vectors undergo batch normalization and ReLU nonlinear transformation, and then are processed by a fully connected (FC) connection to restore multiple vectors to the same dimension. Finally, the multiple vectors are averaged and summed (Avg). Using a classifier, the target category information corresponding to the target brewing object is output.

[0117] In this embodiment, the classifier that outputs target category information can be a Softmax classifier. The Softmax classifier converts the score value obtained from linear classification into a probability value for classification.

[0118] In some embodiments, the first feature vector obtained by convolution is subjected to global max pooling to obtain a third feature vector, and the first feature vector is subjected to global average pooling to obtain a fourth feature vector.

[0119] The fourth and third feature vectors obtained from global average pooling and global max pooling are segmented and truncated to obtain the average pooling feature vector group and the max pooling feature vector.

[0120] It is understandable that the average pooling feature vector group and the max pooling feature vector group each include multiple second feature vectors. The second feature vector refers to the feature vector output after all pooling operations are completed. The multiple second feature vectors in the average pooling feature vector group have different dimensions.

[0121] like Figure 2 As shown, the first feature vector is a vector of size 2048x1x1.

[0122] The average pooling operation reduces the dimension of the first feature vector by truncating 3 / 4, 1 / 2, and 1 / 4 of the first feature vector, respectively, and outputs a1 of 1536 dimensions, b1 of 1024 dimensions, and c1 of 512 dimensions.

[0123] The average pooling feature vector group includes a second feature vector a1 of size 1536, a second feature vector b1 of size 1024, a second feature vector c1 of size 512, and a second feature vector d1 of size 2048 that retains the original dimensions.

[0124] Max pooling reduces the dimension of the first feature vector by truncating 3 / 4, 1 / 2, and 1 / 4 of the first feature vector, respectively, and outputs a2 of size 1536, b2 of size 1024, and c2 of size 512.

[0125] The max-pooling feature vector group includes a second feature vector a2 of size 1536, a second feature vector b2 of size 1024, a second feature vector c2 of size 512, and a second feature vector d2 of size 2048 that retains the original dimensions.

[0126] In this embodiment, the multiple second feature vectors included in the average pooling feature vector group and the max pooling feature vector group are further enriched by batch normalization, activation function and fully connected dimension expansion. The resulting vectors are then averaged and summed to finally output the target category information.

[0127] In this embodiment, by fusing different types of pooling operations, the pooling information of the target image is fused, enriching the overall shape features of the target brewing object, further improving the detail recognition accuracy of the target image, making the target category information corresponding to the identified target brewing object more accurate, and making the temperature of the water dispenser more suitable for the target brewing object.

[0128] In some embodiments, after fusing pooling information of the target image, an attention mechanism is introduced to make the allocation of multiple second feature vectors obtained after pooling more reasonable. Different weight values ​​are assigned to different second feature vectors, making the detail processing of the target brewing object in the target image more targeted and improving the recognition accuracy of target category information.

[0129] The attention mechanism is a resource allocation mechanism that can be understood as redistributing weights to resources that were originally evenly distributed according to the importance of different objects.

[0130] In this embodiment, the pooling operation includes global average pooling and global max pooling. When the attention mechanism is introduced, multiple second weight values ​​and multiple first weight values ​​are obtained for the fourth feature vector and the third feature vector, respectively.

[0131] In the max pooling feature vector group, multiple first weight values ​​are assigned to multiple corresponding second feature vectors. That is, multiple first weight values ​​are multiplied by multiple corresponding second feature vectors respectively, and then the vectors obtained by multiplication are subjected to batch normalization, ReLU nonlinear transformation and fully connected processing.

[0132] In the average pooling feature vector group, multiple second weight values ​​are assigned to multiple corresponding second feature vectors. That is, multiple second weight values ​​are multiplied by multiple corresponding second feature vectors respectively, and then the vectors obtained by multiplication are subjected to batch normalization, ReLU nonlinear transformation and fully connected processing.

[0133] The average summation of multiple vectors after batch normalization, ReLU nonlinear transformation and fully connected processing is performed to output the target category information corresponding to the target brewing object.

[0134] The following section introduces three specific attention mechanisms.

[0135] I. Overall attention mechanism.

[0136] The first feature vector output by the backbone network after feature extraction is a vector of size 2048x1x1. After global max pooling, the max pooled feature vector group includes a second feature vector a2 of size 1536, a second feature vector b2 of size 1024, a second feature vector c2 of size 512, and a second feature vector d2 of size 2048 that retains the original dimensions.

[0137] After global average pooling, the average pooling feature vector group includes a second feature vector a1 of size 1536, a second feature vector b1 of size 1024, a second feature vector c1 of size 512, and a second feature vector d1 of size 2048 that retains the original dimensions.

[0138] like Figure 3 As shown, after global max pooling and global average pooling, two branches are drawn out respectively. Each branch converts the 2048x1x1 dimension vector into a 2048 dimension vector. Then, the 2048 dimension vector is passed through a fully connected layer to output a 4-dimensional vector.

[0139] Each branch performs a sigmoid nonlinear transformation on each component of this 4-dimensional vector to obtain four weight values ​​between 0 and 1.

[0140] The four 0-1 weights obtained in the global max pooling branch are the first weight values, and the four 0-1 weights obtained in the global average pooling branch are the second weight values.

[0141] The weights of 0 to 1 on the two branches are assigned to the corresponding second feature vectors, and the second feature vector of each segment is multiplied by the weight of 0 to 1.

[0142] After batch normalization and ReLU nonlinear transformation, and finally through a fully connected layer, all are restored to 2048-dimensional vectors. Finally, the average sum of the multiple vectors is calculated, and the softmax classifier is used to obtain the classification output of the category information of the target brewing object.

[0143] II. Local attention mechanism with shared weights.

[0144] like Figure 4 As shown, after global max pooling and global average pooling, two branches are drawn out respectively. Each branch divides the 2048x1x1 dimension vector into segments to obtain multiple feature vectors corresponding to the second feature vector in each branch.

[0145] The dimensions of the multiple second feature vectors are 2048, 1536, 1024 and 512 respectively, and the dimensions of the multiple feature vectors corresponding to the second feature vectors in each branch are 2048, 1536, 1024 and 512 respectively.

[0146] For each component of the four feature vectors of different dimensions in each branch, a sigmoid nonlinear transformation is performed. The four 0-1 weights obtained in the global max pooling process of this branch are the first weight values, and the four 0-1 weights obtained in the global average pooling process of this branch are the second weight values.

[0147] The weights of 0 to 1 on the two branches are assigned to the corresponding second feature vectors, and the second feature vector of each segment is multiplied by the weight of 0 to 1.

[0148] After batch normalization and ReLU nonlinear transformation, and finally through a fully connected layer, all are restored to 2048-dimensional vectors. Finally, the average sum of the multiple vectors is calculated, and the softmax classifier is used to obtain the classification output of the category information of the target brewing object.

[0149] III. Local attention mechanism without shared weights.

[0150] like Figure 5 As shown, in the global max pooling process, multiple branches corresponding to the number of second feature vectors are pulled, and the 2048x1x1 dimension vector in each branch is segmented and processed into the dimension of the corresponding second feature vector.

[0151] In addition, multiple branches are also drawn during the global average pooling process, and the 2048x1x1 dimension vector in each branch is segmented and processed into the dimension of the corresponding second feature vector.

[0152] In the global max pooling process, the 2048x1x1 dimension vector in the four branches is segmented into four vectors of 2048, 1536, 1024 and 512 dimensions. Then, a sigmoid non-linear transformation is performed on each component of these four vectors of different dimensions. The component value of each vector is a value between 0 and 1, which is to obtain four first weight values.

[0153] In the global average pooling process, we perform the same operation as in the global max pooling process, which involves pulling multiple branches, resulting in four second weight values.

[0154] In the local attention mechanism that does not share weights, the weight values ​​are obtained by using the components corresponding to multiple vectors. Each component of the multiple vectors is multiplied by the value of the second feature vector of each segment, so that each second feature vector is assigned a corresponding attention weight. This mechanism is more targeted for extracting features of the target brewing object in different dimensions of the target image and can effectively improve the recognition accuracy of target category information.

[0155] In some embodiments, feature fusion can enrich the underlying texture details of the target brewing object, improve the detail recognition accuracy of the target image, make the target category information corresponding to the identified target brewing object more accurate, and make the temperature of the water dispenser more suitable for the target brewing object.

[0156] In this embodiment, the target image can be input into a convolutional layer for multi-layer convolution processing. The image features obtained from the first convolutional layer are input into the second layer for convolution, and so on, finally outputting the first feature vector for pooling processing.

[0157] In this embodiment, the size of the image features obtained from each convolutional layer is made consistent with the size of the image features obtained from the next convolutional layer by downsampling. The two image features are added together with the same number of channels and feature size, and then passed up sequentially. Finally, they are fused and added together with the image features obtained from the last convolutional layer, and then the first feature vector is output after the last convolutional process.

[0158] The following is a specific example.

[0159] like Figure 6 As shown, C1 is the image feature obtained from the first layer convolution of the target image. C1 is convolved with 1x1, and the number of channels becomes the same as that of C2. At the same time, C1 is downsampled or convolved with a stride of 2 so that the size after downsampling is the same as the size of the image feature obtained by C2 after 1x1 convolution.

[0160] The result of C1 downsampling is added to the result of C2 convolution, the result of C2 downsampling is added to the result of C3 convolution, and the result of C3 downsampling is added to the result of C4 convolution, and so on, propagating upwards in this manner.

[0161] C5 is the final convolutional layer. The result of downsampling C4 is added to the result of convolution in C5, and then a 3x3 convolution is performed. The output P5 is used as the first feature vector.

[0162] In some embodiments, upward feature fusion can be performed after downward feature fusion to further enrich the underlying texture details of the target brewing object and improve the detail recognition accuracy of the target image.

[0163] In this embodiment, the size of the image features obtained from each convolutional layer is first made consistent with the size of the image features obtained from the previous convolutional layer by upsampling. The two image features are then added together and passed down sequentially to perform feature fusion.

[0164] The upsampled and fused features are then subjected to convolution and downsampling to make the size of the image features in each layer consistent with the size of the image features in the next layer. The two image features are added together and passed up sequentially for feature fusion. Finally, the feature vector is fused and added with the image features obtained from the last convolution layer, and then processed by the last convolution to output the first feature vector.

[0165] The following is a specific example.

[0166] like Figure 7 As shown, C1 is the image feature obtained by the first layer convolution of the target image. C1 is then convolved with a 1x1 convolution, and the number of channels becomes the same as that of C2. Multiple convolutions are performed to obtain C2 to C5.

[0167] The last layer's C5 is convolved with a 1x1 convolution to get the same number of channels as the previous layer's C4. Then, it is upsampled or deconvolved to get the same size as the previous layer's C4. The upsampled result of C5 is added to the result of the convolution of C4, and so on, and so on, passing down the process.

[0168] After the feature fusion is completed in the downward propagation, the fused features of layer C1 are subjected to 1x1 convolution and downsampling operations, so that they are added to the image features obtained by 1x1 convolution of the fused features of layer C2. This process is repeated in the upward propagation.

[0169] C5 is the final convolutional layer. The features fused from the C4 layer are added together with the result of the 1x1 convolution and downsampling, and then subjected to a 3x3 convolution. The output P5 is used as the first feature vector.

[0170] In some embodiments, step 130, determining the target category information corresponding to the target brewing object, can be achieved through a target classification model.

[0171] In this embodiment, the input to the target classification model is the target image, and the output of the target classification model is the target category information corresponding to the target brewing object.

[0172] Object classification models belong to image classification network models. Object classification models include, but are not limited to, classic classification network models such as VGG, ResNet50, and Inception, as well as lightweight network models such as MobileNet, ShuffleNet, and GhostNet.

[0173] The target classification model is trained using sample images. During the training process, the sample images are used as samples, and the category information corresponding to the sample brewing object in the sample image is used as the sample label.

[0174] In this method, different sample brewing objects can be placed in different cups to obtain a sufficient number of sample images for training the target classification model. The category corresponding to the sample brewing object can be determined by manual annotation.

[0175] The control device for the water dispenser provided in the embodiments of the present invention is described below. The control device for the water dispenser described below can be referred to in correspondence with the control method for the water dispenser described above.

[0176] like Figure 8 As shown, the control device for the water dispenser provided in this embodiment of the invention includes:

[0177] The acquisition module 810 is used to acquire the target cup image, which includes the target brewing object;

[0178] The first processing module 820 is used to perform instance segmentation and cropping processing on the target cup image to obtain the target image;

[0179] The second processing module 830 is used to determine the target category information of the target brewing object based on the target image;

[0180] The third processing module 840 is used to output control commands for controlling the water temperature of the water dispenser based on the target category information.

[0181] According to the control device for a water dispenser provided by the present invention, the position of the target brewing object in the target cup image is determined by instance segmentation, and then the target cup image is cropped to remove interference from the non-target brewing object parts of the target cup image. This makes the recognition of the cropped target image more focused on the details of the target brewing object, improving the recognition accuracy and recognition speed of the target brewing object. After the target category information of the target brewing object is identified, a control command is output to adjust the water temperature of the water dispenser, improve the brewing quality of the target brewing object, and thus improve the user's drinking experience.

[0182] In some embodiments, the second processing module 830 is used to perform convolution processing on the target image to obtain a first feature vector; perform pooling processing on the first feature vector to obtain multiple second feature vectors of different dimensions; and perform classification processing on the multiple second feature vectors to obtain target category information.

[0183] In some embodiments, the second processing module 830 is used to perform global max pooling and global average pooling on the first feature vector to obtain a third feature vector and a fourth feature vector; based on the third feature vector and the fourth feature vector, a max pooling feature vector group and an average pooling feature vector group are obtained; wherein, the max pooling feature vector group includes multiple second feature vectors of different dimensions, and the average pooling feature vector group includes multiple second feature vectors of different dimensions; the max pooling feature vector group and the average pooling feature vector group are classified to obtain target category information.

[0184] In some embodiments, the second processing module 830 is used to extract weights from the third feature vector to obtain multiple first weight values; extract weights from the fourth feature vector to obtain multiple second weight values; assign the multiple first weight values ​​to the multiple second feature vectors corresponding to the max pooling feature vector group; assign the multiple second weight values ​​to the multiple second feature vectors corresponding to the average pooling feature vector group; and perform classification processing on the max pooling feature vector group and the average pooling feature vector group after weight allocation to obtain target category information.

[0185] In some embodiments, the second processing module 830 is used to perform multi-layer convolution processing on the target image, downsample the image features after each layer of convolution processing, fuse the downsampled image features of each layer with the image features obtained by the next layer of convolution processing, and obtain the image features obtained by the last layer of convolution processing, and use them as the first feature vector.

[0186] In some embodiments, the second processing module 830 is used to perform multi-layer convolution processing on the target image, upsampling the image features after each convolution processing layer; fusing the image features after each upsampled layer with the image features obtained from the convolution processing layer above; performing convolution processing on the image features after feature fusion, downsampling the image features after each convolution processing layer; fusing the image features after each downsampled layer with the image features obtained from the convolution processing layer below; and obtaining the image features obtained from the last convolution processing layer, using it as the first feature vector.

[0187] In some embodiments, the second processing module 830 is used to input the target image into the target classification model to obtain the target category information output by the target classification model; wherein, the target classification model is trained using the sample image as the sample and the pre-determined category information corresponding to the sample brewing object in the sample image as the sample label.

[0188] The present invention also provides a water dispenser, including a control device for the water dispenser as described above and a camera electrically connected thereto.

[0189] In this embodiment of the invention, the water dispenser can be an under-sink water purifier, a wall-mounted water dispenser, a countertop water dispenser, a standing water dispenser, a countertop water purifier, a standing water purifier, a sparkling water dispenser, or a commercial water purifier, etc.

[0190] The camera can be installed at the water dispenser's water outlet. When a user takes water from the water dispenser by bringing their cup close to the outlet, the camera captures an image of the cup being used.

[0191] According to the water dispenser provided by the present invention, the position of the target brewing object in the target cup image is determined by instance segmentation, and then the target cup image is cropped to remove interference from the non-target brewing object parts of the target cup image. This makes the recognition of the cropped target image more focused on the details of the target brewing object, improving the recognition accuracy and recognition speed of the target brewing object. After the target category information of the target brewing object is identified, a control command is output to adjust the water temperature of the water dispenser, improve the brewing quality of the target brewing object, and thus improve the user's drinking experience.

[0192] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940. The processor 910, communication interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a control method for the water dispenser. This method includes: acquiring a target cup image, the target cup image including a target brewing object; performing instance segmentation and cropping processing on the target cup image to obtain a target image; determining target category information of the target brewing object based on the target image; and outputting a control command for controlling the water temperature of the water dispenser based on the target category information.

[0193] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] Furthermore, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the water dispenser control method provided in the above-described method embodiments. The method includes: acquiring a target cup image, the target cup image including a target brewing object; performing instance segmentation and cropping processing on the target cup image to obtain a target image; determining target category information of the target brewing object based on the target image; and outputting a control command for controlling the water temperature of the water dispenser based on the target category information.

[0195] On the other hand, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the control method for the water dispenser provided in the above embodiments. The method includes: acquiring a target cup image, the target cup image including a target brewing object; performing instance segmentation and cropping processing on the target cup image to obtain a target image; determining target category information of the target brewing object based on the target image; and outputting a control command for controlling the water temperature of the water dispenser based on the target category information.

[0196] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0197] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0199] The above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Although the invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the invention do not depart from the spirit and scope of the invention and should be covered within the scope of the claims of the invention.

Claims

1. A control method for a water dispenser, characterized in that, include: Acquire a target cup image, the target cup image including a target brewing object; The target cup image is obtained by performing instance segmentation and cropping. Based on the target image, determine the target category information of the target brewing object; Based on the target category information, output a control command for controlling the water temperature of the water dispenser. The step of determining the target category information of the target brewing object based on the target image includes: The target image is convolved to obtain a first feature vector; The first feature vector is pooled to obtain multiple second feature vectors of different dimensions and sizes; The multiple second feature vectors are classified to obtain the target category information; The pooling process of the first feature vector to obtain multiple second feature vectors of different dimensions includes: The first feature vector is subjected to global max pooling and global average pooling respectively to obtain the third feature vector and the fourth feature vector. Based on the third feature vector and the fourth feature vector, the max pooling feature vector set and the average pooling feature vector set are obtained; The max pooling feature vector group includes multiple second feature vectors of different dimensions and sizes, and the average pooling feature vector group includes multiple second feature vectors of different dimensions and sizes. The classification process of the plurality of second feature vectors to obtain the target category information includes: The target category information is obtained by classifying the max-pooling feature vector group and the average-pooling feature vector group.

2. The control method for a water dispenser according to claim 1, characterized in that, After obtaining the max-pooling feature vector set and the average-pooling feature vector set based on the third feature vector and the fourth feature vector, the method further includes: Weights are extracted from the third feature vector to obtain multiple first weight values; Weights are extracted from the fourth feature vector to obtain multiple second weight values; The classification process of the max-pooling feature vector group and the average-pooling feature vector group to obtain the target category information includes: The plurality of first weight values ​​are assigned to the plurality of second feature vectors corresponding to the max-pooling feature vector group; The plurality of second weight values ​​are assigned to the plurality of second feature vectors corresponding to the average pooling feature vector group; The weighted max-pooling feature vector group and the average-pooling feature vector group are classified to obtain the target category information.

3. The control method for a water dispenser according to claim 1, characterized in that, The step of performing convolution processing on the target image to obtain the first feature vector includes: The target image is subjected to multi-layer convolution processing, and the image features after each layer of convolution processing are downsampled; The image features after downsampling at each layer are fused with the image features obtained by the convolutional processing at the next layer. The image features obtained from the last convolutional layer are used as the first feature vector.

4. The control method for a water dispenser according to claim 1, characterized in that, The step of performing convolution processing on the target image to obtain the first feature vector includes: The target image is subjected to multi-layer convolution processing, and the image features after each layer of convolution processing are upsampled. The image features upsampled at each layer are fused with the image features obtained from the convolutional processing of the previous layer. The image features after feature fusion are convolved, and the image features after each convolutional layer are downsampled. The image features after downsampling at each layer are fused with the image features obtained by the convolutional processing at the next layer. The image features obtained from the last convolutional layer are used as the first feature vector.

5. The control method for a water dispenser according to any one of claims 1-4, characterized in that, The step of determining the target category information of the target brewing object based on the target image includes: The target image is input into the target classification model to obtain the target category information output by the target classification model; The target classification model is trained using sample images as samples and pre-determined category information corresponding to the sample brewing objects in the sample images as sample labels.

6. A control device for a water dispenser, characterized in that, include: The acquisition module is used to acquire a target cup image, wherein the target cup image includes a target brewing object; The first processing module is used to perform instance segmentation and cropping processing on the target cup image to obtain the target image; The second processing module is used to determine the target category information of the target brewing object based on the target image; The third processing module is used to output control commands for controlling the water temperature of the water dispenser based on the target category information. The second processing module is used to perform convolution processing on the target image to obtain a first feature vector; to perform pooling processing on the first feature vector to obtain multiple second feature vectors of different dimensions; and to perform classification processing on the multiple second feature vectors to obtain target category information. The second processing module is used to perform global max pooling and global average pooling on the first feature vector to obtain a third feature vector and a fourth feature vector; based on the third feature vector and the fourth feature vector, a max pooling feature vector group and an average pooling feature vector group are obtained; wherein, the max pooling feature vector group includes multiple second feature vectors of different dimensions, and the average pooling feature vector group includes multiple second feature vectors of different dimensions; the max pooling feature vector group and the average pooling feature vector group are classified to obtain target category information.

7. A water dispenser, characterized in that, include: A camera, used to capture images of the target cup; The control device for the water dispenser as described in claim 6 is electrically connected to the camera.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the control method for the water dispenser as described in any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for the water dispenser as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for the water dispenser as described in any one of claims 1 to 5.

Images