Virtual pet adjustment method, device, electronic device and storage medium

Abstract

The present invention discloses a method, device, electronic device, and storage medium for adjusting a virtual pet. The method is applied to a virtual pet adjustment system, comprising: a virtual pet adjustment device and a wearable device; the multiple wearable devices maintaining a communication connection with the virtual pet adjustment device; the method comprising: the virtual pet adjustment device receiving a user's body parameters from the wearable device; the virtual pet adjustment device determining a target appearance and a target body shape that match the virtual pet based on the body parameters and the type of the user's virtual pet; and the virtual pet adjustment device adjusting the virtual pet's appearance to the target appearance and body shape to the target body shape.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for adjusting a virtual pet. Background Art

[0002] With the booming pet market, nearly every household wants to keep a pet. However, practically, pet ownership presents challenges, such as busy work schedules, relatively short lifespans, and restrictions on pet ownership in certain areas. These factors prevent many would-be pet owners from achieving their goals. Therefore, virtual pets offer the following advantages: long lifespans and unlimited growth; they can be raised anytime, anywhere, without requiring specialized feeding or care; and they are not restricted by living environments. Users simply need to install the appropriate app and operate their virtual pets online.

[0003] However, in the current process of raising virtual pets, the appearance and body shape of the virtual pets need to be changed and adjusted by the users themselves, and the interaction with the users is relatively simple. Summary of the Invention

[0004] The embodiments of the present application provide a virtual pet adjustment method, device, electronic device and storage medium, which obtain the user's body parameters through a wearable device, thereby automatically adjusting the appearance and body shape of the virtual pet based on the user's body parameters without the need for manual adjustment by the user.

[0005] In a first aspect, an embodiment of the present application provides a virtual pet adjustment method, which is applied to a virtual pet adjustment system, wherein the virtual pet adjustment system includes: a virtual pet adjustment device and a wearable device; the plurality of wearable devices maintain a communication connection with the virtual pet adjustment device; the method includes:

[0006] The virtual pet adjustment device receives the user's body parameters from the wearable device;

[0007] The virtual pet adjustment device determines a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet;

[0008] The virtual pet adjustment device adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

[0009] In a second aspect, an embodiment of the present application provides a virtual pet adjustment device, comprising: a transceiver unit and a processing unit;

[0010] The transceiver unit is configured to receive the user's body parameters from the wearable device;

[0011] The processing unit is configured to determine a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet;

[0012] The virtual pet adjustment device adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor being connected to a memory, the memory being used to store a computer program, the processor being used to execute the computer program stored in the memory, so that the electronic device executes the method described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to enable the computer to execute the method described in the first aspect.

[0016] The implementation of the embodiments of the present application has the following beneficial effects:

[0017] It can be seen that in the embodiment of the present application, the virtual pet adjustment device can receive body parameters from the wearable device, and then determine the target appearance and target body shape that match the virtual pet based on the body parameters and the type of the user's virtual pet, and adjust the appearance of the virtual pet to the target appearance, and adjust the body shape of the virtual pet to the target body shape, thereby realizing automatic adjustment of the appearance of the virtual pet based on the user's body parameters, without the user actively adjusting, and realizing intelligent and convenient adjustment of the appearance of the virtual pet. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 A schematic diagram of a virtual pet adjustment system provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of a virtual pet adjustment method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of constructing a first characteristic matrix provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of constructing a second characteristic matrix provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of determining a weight matrix provided in an embodiment of the present application;

[0024] Figure 6 A flowchart of a method for generating a virtual pet provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of target detection provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of a multi-task model provided in an embodiment of the present application;

[0027] Figure 9 A block diagram of the functional units of a virtual pet adjustment device provided in an embodiment of the present application;

[0028] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0031] References herein to "embodiments" mean that a particular feature, result, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] See Figure 1 , Figure 1 A schematic diagram of a virtual pet adjustment system provided in an embodiment of the present application. The virtual adjustment system includes a virtual pet adjustment device 10 and a wearable device 20. The virtual pet adjustment device 10 and the wearable device 20 maintain a communication connection, a user's virtual pet is generated on the virtual pet adjustment device 10, and there are one or more wearable devices 20.

[0033] Exemplarily, the virtual pet adjustment device 10 receives the user's body parameters from the wearable device; then, based on the body parameters and the type of the user's virtual pet, determines the target appearance and target body shape that match the virtual pet; finally, adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

[0034] It can be seen that in the embodiment of the present application, the virtual pet adjustment device can receive body parameters from the wearable device, and then determine the target appearance and target body shape that match the virtual pet based on the body parameters and the type of the user's virtual pet, and adjust the appearance of the virtual pet to the target appearance, and adjust the body shape of the virtual pet to the target body shape, thereby realizing automatic adjustment of the appearance of the virtual pet based on the user's body parameters, without the user actively adjusting, and realizing intelligent and convenient adjustment of the appearance of the virtual pet.

[0035] See Figure 2 , Figure 2 This is a flow chart of a virtual pet adjustment method provided in an embodiment of the present application. The method is applied to the virtual pet adjustment device 10 described above. The method includes but is not limited to the following steps:

[0036] 201: The virtual pet adjustment apparatus receives the user's body parameters from the wearable device.

[0037] Optionally, when the virtual pet adjustment device detects a preset event, it sends an instruction to the wearable device, thereby instructing the wearable device to report the user's body parameters to the virtual pet adjustment device through the instruction.

[0038] Exemplarily, the preset event may be that the user is in a preset range or a preset operation is detected.

[0039] In one embodiment of the present application, when detecting that the current moment is within a preset time period, the virtual pet adjustment device sends a first instruction to the wearable device, wherein the first instruction is used to instruct the wearable device to report the user's location information to the virtual pet adjustment device. Then, a response message to the first instruction is received from the wearable device, wherein the response message includes the user's location information. When detecting that the location information is within a preset range, a second instruction is sent to the wearable device, wherein the second instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device. In response to the second instruction, the wearable device obtains the user's physical parameters and reports them to the virtual pet adjustment device.

[0040] It should be noted that the preset time period is determined based on the user's historical habits, and the probability of the user operating the virtual pet within the preset time period is greater than a threshold. For example, the user's historical times of operating the virtual pet are recorded, and the preset time period is determined based on these historical times. For example, if statistics show that the user generally operates the virtual pet between 8:00 PM and 10:00 PM, then 8:00 PM to 10:00 PM is set as the preset time period. Optionally, the preset range is a pre-defined area within which the user can operate the virtual pet. For example, if the virtual pet adjustment device is in the bedroom, the user can operate the virtual pet on the virtual pet adjustment device's visual interface, so the bedroom can be used as the preset range. Therefore, if it is determined that the current time is within the preset time period, meaning that there is a high probability that the user will operate the virtual pet, the wearable device is requested to report the user's location information. If it is further determined that the user is within the preset range, meaning that the user can view or operate the virtual pet, the wearable device is requested to report the user's physical parameters. The virtual pet's appearance can then be automatically adjusted based on the user's physical parameters.

[0041] In another embodiment of the present application, the virtual pet adjustment device obtains a user's start-up request for the virtual pet, that is, detects that the user starts the virtual pet, for example, calls out the virtual pet, etc.; in response to the start-up request, a third instruction is sent to the wearable device, and the third instruction is used to instruct the wearable device to report the user's body parameters to the virtual pet adjustment device.

[0042] It should be noted that the aforementioned body parameters include multiple first parameter values ​​of the user's multiple physiological parameters at each sampling moment, as well as second parameter values ​​of the motion parameters at each sampling moment. In other words, when the virtual pet adjustment device instructs the wearable device to perform body parameter measurement, the wearable device will collect the user's body parameters at a preset sampling interval and report the multiple first parameter values ​​and second parameter values ​​collected at each sampling moment to the wearable device.

[0043] Optionally, the multiple physiological parameters may be the user's heart rate (or pulse), respiratory rate, body temperature, blood pressure, blood concentration, carbon dioxide concentration, blood glucose content, skin moisture, galvanic skin response index, and bioresistance, etc. The motion parameter is the user's walking speed, where the walking speed is the ratio of the number of steps in a sampling interval to the duration of the sampling interval. In addition, the wearable device of the present application may be one or more, and the present application is mainly described by taking one wearable device as an example.

[0044] Therefore, for physiological parameters, when a sampling moment arrives, the currently collected physiological parameter value can be used as the first parameter value at that sampling moment; for motion parameters, when a sampling moment arrives, the number of motion steps between that sampling moment and the next sampling moment can be obtained, and the ratio of the number of motion steps to the duration of the sampling interval can be used as the second parameter value corresponding to that sampling moment. In this way, multiple first parameter values ​​and second parameter values ​​corresponding to each sampling moment can be obtained.

[0045] 202: The virtual pet adjustment device determines a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet.

[0046] Exemplarily, the user's emotions are determined based on physical parameters, where the user's emotions include but are not limited to: happy, sad, excited, depressed, sad, etc.; based on the correspondence between the user's emotions, the type, appearance and body shape of the virtual pet, the target appearance and target body shape that match the virtual pet are determined.

[0047] Table 1 shows the corresponding relationships among emotions, types, appearances, and sizes of virtual pets.

[0048]

[0049] Exemplarily, multiple first parameter values ​​of multiple physiological parameters at each sampling moment are vectorized to obtain multiple first eigenvectors at each sampling moment, wherein each physiological parameter corresponds to a first parameter value at each sampling moment, and a first eigenvector at each sampling moment. That is, the first parameter value of each physiological parameter at each moment is mapped to obtain the first eigenvector of each physiological parameter at each moment; all the first eigenvectors at multiple sampling moments are concatenated to obtain a first characteristic matrix, that is, the first eigenvectors of each physiological parameter at each sampling moment are concatenated to obtain the first characteristic matrix. The length of the first characteristic matrix is ​​the number of sampling moments, the width is the number of physiological parameters, and the height is the dimension of each first eigenvector.

[0050] For example, if Figure 3 As shown, the physiological parameters include S1, S2, S3, and S4, and the sampling times are T1, T2, T3, T4, and T5. First, the first parameter value of each physiological parameter at each sampling time is vectorized to obtain the first eigenvector of each physiological parameter at each sampling time, as shown in Figure 3 As shown in , the first eigenvector of the physiological parameter S4 at the sampling time T1 can be obtained. Then, the first eigenvectors of the physiological parameters S1, S2, S3, and S4 at the time T1, T2, T3, T4, and T5 can be spliced ​​to obtain Figure 3 As shown in the first characteristic matrix, the length of the first characteristic matrix is ​​5, the width is 4, and the height is the dimension of the first characteristic vector.

[0051] Furthermore, the second parameter value of the motion parameter at each sampling moment is vectorized to obtain the second eigenvector at each sampling moment. That is, the second parameter value of the motion parameter at each sampling moment is vectorized to obtain the second eigenvector at each sampling moment, wherein the dimensions of the multiple first eigenvectors at each sampling moment are the same as the dimensions of the second eigenvector at each sampling moment. Then, the multiple second eigenvectors at multiple sampling moments are concatenated to obtain a second characteristic matrix, wherein the length of the second characteristic matrix is ​​the number of sampling moments, the width is the number of motion parameters, and the height is the dimension of the second eigenvector.

[0052] For example, if Figure 4 As shown in the figure, the second parameter value of the motion parameter Y at the sampling time T1 is vectorized to obtain the second eigenvector of the motion parameter Y at the sampling time T1. Then, the second eigenvectors of the motion parameter Y at the times T1, T2, T3, T4, and T5 are concatenated to obtain the second characteristic matrix.

[0053] Furthermore, a weight matrix is ​​determined based on the first feature matrix and the second feature matrix. Specifically, the similarity between the first eigenvector of each physiological parameter in the first feature matrix at each sampling moment and the second eigenvector of each physiological parameter in the second feature matrix at each sampling moment is determined; multiple similarities of the multiple physiological parameters at each sampling moment are normalized to obtain weight coefficients of the multiple physiological parameters at each sampling moment; and the weight coefficients of the multiple physiological parameters at each sampling moment are vertically replicated to obtain the weight matrix.

[0054] like Figure 5As shown, first determine the similarity between the first eigenvector of each physiological parameter in the first feature matrix at each sampling moment and the second eigenvector in the second feature matrix at each sampling moment, that is, take the Euclidean distance between the first eigenvector and the second eigenvector as the similarity between the two; then, normalize the similarity between the second eigenvector at each sampling moment and the first eigenvector at the same sampling moment, and obtain the weight coefficients of multiple physiological parameters at each sampling moment. Figure 5 As shown, the weight coefficients of multiple physiological parameters at sampling time T1 are α11, α21, α31, and α41 respectively; the weight coefficients at sampling time T2 are α12, α22, α32, and α42 respectively; the weight coefficients at sampling time T3 are α13, α23, α33, and α43 respectively; the weight coefficients at sampling time T4 are α14, α24, α34, and α44 respectively; and the weight coefficients at sampling time T5 are α15, α25, α35, and α45 respectively. Then, the weight coefficient matrix is ​​copied vertically to obtain a weight matrix, in which the values ​​of each column in the weight matrix are the same, that is, the weight corresponding to each physiological parameter at each sampling time is the same, and they are all the weight coefficients of the physiological parameter at each sampling time. As shown Figure 5 As shown, the weights of the physiological parameter S1 at the sampling time T1 are all the same, and are all α11.

[0055] Furthermore, the first characteristic matrix is ​​weighted according to the weight matrix to obtain a third characteristic matrix, that is, the weight matrix is ​​subjected to matrix dot multiplication with the first characteristic matrix, that is, corresponding elements are multiplied to obtain the third characteristic matrix.

[0056] It should be noted that since the motion parameters reflect the psychological characteristics of the user during the sampling period, the similarity between the motion parameters (second eigenvector) and the physiological parameters (first eigenvector) is calculated, which is similar to attention processing of the motion parameters and the physiological parameters. Finally, the weights obtained by the attention processing are weighted on the motion parameters, so that the physiological parameters in the third characteristic matrix can better reflect the user's emotions, and thus the final determined emotion accuracy is relatively high.

[0057] Finally, emotion recognition is performed based on the third feature matrix to obtain the user's emotion. Exemplarily, the third feature matrix is ​​input into a classifier for feature extraction and classification, thereby obtaining the user's emotion, wherein the classifier is pre-trained.

[0058] In one embodiment of the present application, it is first explained that a user's physical parameters are not fixed but can change in response to emotional fluctuations. Different physical parameter values ​​can correspond to different emotional states. Therefore, a mapping relationship between different physical parameter values ​​and emotions is pre-established. After obtaining the user's physical parameters, the user's emotions can be determined based on the user's physical parameters and this mapping relationship.

[0059] Specifically, based on the body parameters collected at each sampling moment (i.e., the multiple physiological parameters and motion parameters mentioned above), the emotion at each sampling moment is determined based on the above mapping relationship and the body parameters collected at each sampling moment. Optionally, voting is performed based on multiple emotions at multiple sampling moments, and the emotion with the largest number of votes at the multiple sampling moments is used as the emotion of the user. Optionally, multiple emotions at multiple sampling moments can also be fitted into an emotion curve; then, the emotion curve is fitted with the curve template of each emotion to obtain a fitting error, and the emotion corresponding to the curve template of the fitting error is used as the emotion of the above user.

[0060] 203: The virtual pet adjusting device adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

[0061] Exemplarily, the current appearance of the virtual pet is obtained. If the current appearance is different from the target appearance, the target texture corresponding to the target appearance is first obtained. If the current appearance is the same as the target appearance, the texture corresponding to the current appearance is directly used as the target texture. Furthermore, after determining the target texture, the body shape adapted by the target texture is obtained. If the body shape matches the target body shape, the target texture is directly used to overwrite the texture corresponding to the current appearance, thereby adjusting the appearance of the virtual pet to the target appearance and the body shape to the target body shape; if the body shape does not match the target body shape, the target texture is scaled, and then the scaled target texture is used to overwrite the texture corresponding to the current appearance, thereby adjusting the appearance of the virtual pet to the target appearance and the body shape to the target body shape.

[0062] It can be seen that in the embodiment of the present application, the virtual pet adjustment device can receive body parameters from the wearable device, and then determine the target appearance and target body shape that match the virtual pet based on the body parameters and the type of the user's virtual pet, and adjust the appearance of the virtual pet to the target appearance, and adjust the body shape of the virtual pet to the target body shape, thereby realizing automatic adjustment of the appearance of the virtual pet based on the user's body parameters, without the user actively adjusting, and realizing intelligent and convenient adjustment of the appearance of the virtual pet.

[0063] In one embodiment of the present application, after determining the user's emotion, voice parameters corresponding to the virtual pet can be determined based on the user's emotion; the corresponding conversation content with the user can be obtained based on the user's emotion; a reminder time can be determined based on the user's emotion; and then, at the reminder time, the virtual pet can be controlled to conduct the conversation with the user using the voice parameters. For example, a correspondence between emotion and voice parameters can be pre-set. Therefore, after determining the user's emotion, the user's voice parameters can be determined based on this correspondence. Similarly, a correspondence between emotion and conversation content can be pre-set. Therefore, after determining the user's emotion, the user's conversation content can be determined based on this correspondence. Finally, a correspondence between duration and emotion can be pre-set. Therefore, after determining the emotion, a preset duration can be determined based on this correspondence. The moment corresponding to the preset duration after the emotion was determined is then used as the reminder time. This enables intelligent conversation with the user at the reminder time and intelligent regulation of the user's emotion.

[0064] See Figure 6 , Figure 6 This is a flow chart of a method for generating a virtual pet provided in an embodiment of the present application. The method is applied to the aforementioned virtual pet adjustment device 10. The method includes but is not limited to the following steps:

[0065] 601: The virtual pet adjustment device obtains a plurality of pet images.

[0066] The multiple pet images are pet images taken by the user at multiple shooting angles of the physical pet, and the multiple shooting angles correspond one-to-one to the multiple pet images.

[0067] Optionally, the multiple shooting angles of the present application include but are not limited to: front shooting angle, rear shooting angle, left shooting angle, right shooting angle and overhead shooting angle.

[0068] 602: The virtual pet adjustment device extracts features from the multiple pet images to obtain features of each body part of the physical pet.

[0069] Exemplarily, target detection is performed on the pet image corresponding to each shooting angle to obtain the body parts of the physical pet captured at each shooting angle.

[0070] Optionally, end-to-end target detection can be used to perform target detection on the pet image corresponding to each shooting angle to obtain the target in the pet image corresponding to each shooting angle and the category of the target, thereby obtaining the body parts of the physical pet captured at each shooting angle.

[0071] like Figure 7 As shown, feature extraction is performed on the above-mentioned pet image to obtain a first feature map; the first feature map is tiled to obtain multiple first vectors; the multiple first vectors are respectively fused with position codes to obtain multiple second vectors, wherein the position codes are obtained by training; the multiple second vectors are input into the encoder for encoding to obtain multiple third vectors; then, according to each shooting angle and the type of the physical pet, multiple query vectors corresponding to each shooting angle are obtained, wherein the multiple query vectors are pre-trained, and each query vector is used to characterize the characteristics of a body part of the above-mentioned type of physical pet at the shooting angle, for example, a query vector at a frontal shooting angle is used to characterize the characteristics of this type of physical pet when photographed at a frontal shooting angle; then, attention processing is performed on the multiple third vectors and the multiple query vectors to obtain multiple fourth vectors, wherein the attention processing can be cross-attention processing or multi-head attention processing, etc., and this application does not limit the attention method; finally, target detection is performed based on the multiple fourth vectors to obtain the body part of the above-mentioned physical pet photographed at each shooting angle.

[0072] Exemplarily, target detection is performed based on the multiple fourth vectors to predict multiple first candidate frames and multiple first categories, where the multiple first candidate frames correspond one-to-one to the multiple first categories. Confidence checks are then performed on the multiple first candidate frames and the multiple first categories, and target first candidate frames are screened from the multiple first candidate frames, where the number of target first candidate frames is one or more. The first category corresponding to the target first candidate frame is used as the body part of the physical pet captured at each shooting angle.

[0073] It should be noted that different shooting angles capture different body parts. For example, a frontal shooting angle can capture a pet's mouth and nose, while a left profile shooting angle can capture its left profile. Furthermore, different pets also have different body parts at the same shooting angle. Therefore, setting corresponding query vectors for each pet at different shooting angles improves target detection efficiency and accuracy. Without setting corresponding query vectors, the same set of query vectors would be used for pet images from each shooting angle. In this case, some pet images from certain shooting angles would have multiple, invalid matches with the query vectors, resulting in relatively low target recognition efficiency and accuracy. For example, without setting corresponding query vectors, to ensure comprehensive target recognition, a query vector corresponding to the mouth must be included in the query vector. Therefore, when detecting targets from left, right, and overhead angles, the query vector for the mouth must be matched, resulting in relatively low target detection efficiency and accuracy from these shooting angles.

[0074] Furthermore, feature extraction is performed on the pet image corresponding to each shooting angle to obtain features of the body part captured at each shooting angle. For example, the features of the body part captured at each shooting angle include, but are not limited to, the size, hair sparseness, and hair color of the body part. The size can be represented by the area of ​​the body part in the pet image.

[0075] Exemplarily, for the i-th body part, an image to be processed corresponding to the i-th body part is captured from the pet image corresponding to each shooting angle, wherein the i-th body part is any one of the body parts captured at each shooting angle, that is, the image selected by the first candidate frame of the target is captured from the pet image to obtain the image to be processed corresponding to the i-th body part; the image to be processed is input into the multi-task model for feature extraction to obtain the features corresponding to the i-th body part; based on the features corresponding to the i-th body part, the features of the body part captured at each shooting angle are determined. It should be noted that each body part captured at each shooting angle is processed in the same manner as the i-th body part to obtain the features of each body part captured at each shooting angle.

[0076] For example, Figure 8 As shown, the multi-task model includes multiple feature extraction networks (such as Figure 8 Feature extraction network 1, feature extraction network 2, ..., feature extraction network n), multiple weight setting networks (such as Figure 8 The weight setting network 1 and the weight setting network 2 shown) and the multiple task prediction networks (such as Figure 8 Task prediction network 1 and task prediction network 2 are shown), wherein multiple weight setting networks and multiple task prediction networks correspond one to one.

[0077] It should be noted that each feature extraction network can be composed of multiple convolutional layers, which mainly perform feature extraction on pet images, for example, extracting features such as contours and textures in pet images. Each service prediction network can be composed of a multi-layer perceptron, and each weight setting network can be composed of a softmax classifier.

[0078] Further, based on Figure 8 The multi-task model shown can perform feature extraction on the image to be processed through multiple feature extraction networks, and obtain the feature extraction results corresponding to each feature extraction network, that is, obtain the feature vector of the image to be processed; then, the image to be processed is classified through each weight setting network, that is, feature extraction is performed first, and then softmax classification is performed, and the classification results corresponding to each weight setting network are obtained, that is, the probability of falling into each feature extraction network.

[0079] Furthermore, based on the classification results corresponding to each weight setting network, the weight corresponding to each feature extraction network is determined when the classification processing is performed by each weight setting network. For example, the probability of falling into each feature extraction network can be directly used as the weight of each feature extraction network.

[0080] Furthermore, according to the weight corresponding to each feature extraction network when performing classification processing through each weight setting network, the feature extraction results corresponding to the multiple task prediction networks are weighted to obtain the input data of the task prediction network corresponding to each weight setting network.

[0081] Optionally, in one embodiment of the present application, after obtaining the weighted result corresponding to each weight setting network, the weighted result is not directly used as the input data of the task prediction network corresponding to each weight setting network, such as Figure 8 Instead, the weighted results of each weight setting network are concatenated with the type feature vector to obtain a target feature vector corresponding to each weight setting network. The type feature vector is obtained by vectorizing the type of the entity pet. The target feature vector corresponding to each weight setting network is used as the input data for the task prediction network corresponding to each weight setting network.

[0082] It should be noted that the type feature vector is spliced ​​into the input data of each task prediction network, that is, the type information of the entity pet is spliced. In this way, each task prediction network can know the type of the entity pet in advance when performing feature prediction. Since the type of the entity pet can indicate to a certain extent what body parts the entity pet has at each shooting angle and what the characteristics of the body parts are, it is equivalent to inputting prior information into the task prediction network, which can improve the prediction accuracy of each task prediction network for feature prediction.

[0083] Furthermore, through each task prediction network, task prediction is performed on the input data of each task prediction network to obtain the task prediction result corresponding to each task prediction network, that is, a feature of the body part is obtained, wherein different task prediction networks are used to predict different features. For example, a task prediction network is used to predict hair color, and another task prediction network is used to predict hair sparseness.

[0084] According to the task prediction results corresponding to each task prediction network, the feature corresponding to the i-th body part is obtained, that is, the task prediction results corresponding to each task prediction network are combined to obtain the feature corresponding to the i-th body part.

[0085] Finally, the features of the body parts captured from multiple shooting angles are fused to obtain the features of each body part of the physical pet. Specifically, the features of the body parts captured from each shooting angle are merged to obtain the features of each body part of the physical pet. For example, the features of the mouth captured from a frontal angle are merged with the features of the back captured from a top-down angle to obtain the features of the mouth and back of the physical pet.

[0086] 603: The virtual pet adjustment device generates a virtual pet corresponding to the physical pet according to the characteristics of each body part of the physical pet.

[0087] For example, based on the characteristics of each body part of the physical pet, a corresponding pet sticker is determined for each body part. Specifically, based on the mapping relationship between the body part characteristics and the pet sticker, a corresponding pet sticker is determined for each body part of the physical pet. For example, if the pet is a cat and the body part is a head, and the head is black and white, a pet sticker capable of generating a black and white cat can be obtained from a pet sticker library, where each sticker in the pet sticker library is preconfigured. Finally, the pet stickers corresponding to each body part are superimposed to generate a virtual pet corresponding to the physical pet.

[0088] See Figure 9 , Figure 9 The embodiment of the present application provides a block diagram of the functional units of a virtual pet adjustment device. The virtual pet adjustment device 900 includes: a transceiver unit 901 and a processing unit 902, wherein:

[0089] The transceiver unit 901 is configured to receive the user's body parameters from the wearable device;

[0090] Processing unit 902, configured for the virtual pet adjustment device to determine a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet;

[0091] The appearance of the virtual pet is adjusted to the target appearance, and the body shape of the virtual pet is adjusted to the target body shape.

[0092] In some possible implementations, in determining a target appearance and a target body shape that matches the virtual pet based on the body parameters and the type of the user's virtual pet, the processing unit 902 is specifically configured to:

[0093] determining the user's emotion based on the physical parameter;

[0094] According to the corresponding relationship between the emotion of the user, the type, appearance and body shape of the virtual pet, a target appearance and body shape matching the virtual pet are determined.

[0095] In some possible implementations, the body parameters include multiple first parameter values ​​of multiple physiological parameters at each sampling moment and second parameter values ​​of motion parameters at each sampling moment; in determining the user's emotion based on the body parameters, the processing unit 902 is specifically configured to:

[0096] Vectorizing the multiple first parameter values ​​of the multiple physiological parameters at each sampling moment to obtain multiple first eigenvectors at each sampling moment, wherein each physiological parameter corresponds to one first eigenvector at each sampling moment;

[0097] All the first eigenvectors at multiple sampling moments are concatenated to obtain a first eigenmatrix;

[0098] Vectorizing a second parameter value of the motion parameter at each sampling moment to obtain a second eigenvector at each sampling moment, wherein the dimensions of the plurality of first eigenvectors at each sampling moment and the second eigenvector at each sampling moment are the same;

[0099] splicing the plurality of second eigenvectors at the plurality of sampling moments to obtain a second eigenmatrix;

[0100] Determining a weight matrix based on the first feature matrix and the second feature matrix;

[0101] Performing weighted processing on the first characteristic matrix according to the weight matrix to obtain a third characteristic matrix;

[0102] Emotion recognition is performed based on the third feature matrix to obtain the user's emotion.

[0103] In some possible implementations, in determining a weight matrix based on the first feature matrix and the second feature matrix, the processing unit 902 is specifically configured to:

[0104] respectively determining the similarity between a first eigenvector of each physiological parameter in the first feature matrix at each sampling moment and a second eigenvector of each physiological parameter in the second feature matrix at each sampling moment;

[0105] Normalizing the multiple similarities of the multiple physiological parameters at each sampling moment to obtain weight coefficients of the multiple physiological parameters at each sampling moment;

[0106] The weight coefficients of the multiple physiological parameters at each sampling moment are longitudinally replicated and combined to obtain the weight matrix.

[0107] In some possible implementations, after determining the user's emotion based on the physical parameters, the processing unit 902 is further configured to:

[0108] determining voice parameters corresponding to the virtual pet based on the user's emotions;

[0109] Acquiring conversation content corresponding to the user based on the user's emotion;

[0110] Determining a reminder time according to the user's emotion;

[0111] At the reminder moment, the virtual pet is controlled to have the conversation with the user through the voice parameters.

[0112] In some possible implementations, before receiving the user's physical parameters from the wearable device, when the processing unit 902 detects that the current moment is within a preset time period, the transceiver unit 901 is further configured to send a first instruction to the wearable device, wherein the first instruction is configured to instruct the wearable device to report the user's location information to the virtual pet adjustment device; and receive a response message to the first instruction from the wearable device, wherein the response message includes the user's location information.

[0113] When the processing unit 902 detects that the location information is within a preset range, the transceiver unit 901 is further configured to send a second instruction to the wearable device, where the second instruction is configured to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device.

[0114] In some possible implementations, before receiving the user's physical parameters from the wearable device, the transceiver unit 901 is further used to obtain the user's start-up request for the virtual pet; in response to the start-up request, send a third instruction to the wearable device, and the third instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device.

[0115] See Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 As shown, the electronic device 1000 includes a transceiver 1001, a processor 1002, and a memory 1003. These are connected via a bus 1004. The memory 1003 is used to store computer programs and data, and can transmit the data stored in the memory 1003 to the processor 1002.

[0116] The processor 1002 is configured to read the computer program in the memory 1003 and perform the following operations:

[0117] Controlling the transceiver 1001 to receive the user's body parameters from the wearable device;

[0118] The virtual pet adjustment device determines a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet;

[0119] The appearance of the virtual pet is adjusted to the target appearance, and the body shape of the virtual pet is adjusted to the target body shape.

[0120] In some possible implementations, in determining a target appearance and a target body shape that matches the virtual pet based on the body parameters and the type of the user's virtual pet, the processor 1002 is specifically configured to perform the following operations:

[0121] determining the user's emotion based on the physical parameter;

[0122] According to the corresponding relationship between the emotion of the user, the type, appearance and body shape of the virtual pet, a target appearance and body shape matching the virtual pet are determined.

[0123] In some possible implementations, the body parameters include multiple first parameter values ​​of multiple physiological parameters at each sampling moment and second parameter values ​​of motion parameters at each sampling moment; in determining the user's emotion based on the body parameters, the processor 1002 is specifically configured to perform the following operations:

[0124] Vectorizing the multiple first parameter values ​​of the multiple physiological parameters at each sampling moment to obtain multiple first eigenvectors at each sampling moment, wherein each physiological parameter corresponds to one first eigenvector at each sampling moment;

[0125] All the first eigenvectors at multiple sampling moments are concatenated to obtain a first eigenmatrix;

[0126] Vectorizing a second parameter value of the motion parameter at each sampling moment to obtain a second eigenvector at each sampling moment, wherein the dimensions of the plurality of first eigenvectors at each sampling moment and the second eigenvector at each sampling moment are the same;

[0127] splicing the plurality of second eigenvectors at the plurality of sampling moments to obtain a second eigenmatrix;

[0128] Determining a weight matrix based on the first feature matrix and the second feature matrix;

[0129] Performing weighted processing on the first characteristic matrix according to the weight matrix to obtain a third characteristic matrix;

[0130] Emotion recognition is performed based on the third feature matrix to obtain the user's emotion.

[0131] In some possible implementations, in determining a weight matrix based on the first feature matrix and the second feature matrix, the processor 1002 is specifically configured to perform the following operations:

[0132] respectively determining the similarity between a first eigenvector of each physiological parameter in the first feature matrix at each sampling moment and a second eigenvector of each physiological parameter in the second feature matrix at each sampling moment;

[0133] Normalizing the multiple similarities of the multiple physiological parameters at each sampling moment to obtain weight coefficients of the multiple physiological parameters at each sampling moment;

[0134] The weight coefficients of the multiple physiological parameters at each sampling moment are longitudinally replicated and combined to obtain the weight matrix.

[0135] In some possible implementations, after determining the user's emotion based on the physical parameters, the processor 1002 is further configured to perform the following operations:

[0136] determining voice parameters corresponding to the virtual pet based on the user's emotions;

[0137] Acquiring conversation content corresponding to the user based on the user's emotion;

[0138] Determining a reminder time according to the user's emotion;

[0139] At the reminder moment, the virtual pet is controlled to have the conversation with the user through the voice parameters.

[0140] In some possible implementations, before receiving the user's physical parameters from the wearable device, the processor 1002 is further configured to perform the following operations:

[0141] When it is detected that the current moment is within a preset time period, the control transceiver 1001 sends a first instruction to the wearable device, wherein the first instruction is used to instruct the wearable device to report the user's location information to the virtual pet adjustment device; and receives a response message to the first instruction from the wearable device, wherein the response message includes the user's location information.

[0142] When it is detected that the position information is within a preset range, the control transceiver 1001 sends a second instruction to the wearable device, where the second instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device.

[0143] In some possible implementations, before receiving the user's physical parameters from the wearable device, the processor 1002 is further configured to perform the following operations:

[0144] The control transceiver 1001 obtains the user's start-up request for the virtual pet; in response to the start-up request, the control transceiver 1001 sends a third instruction to the wearable device, and the third instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device.

[0145] Specifically, the transceiver 1001 may be Figure 9 The transceiver unit 901 of the virtual pet adjustment device 900 of the embodiment described above, the processor 1002 may be Figure 9 The processing unit 902 of the virtual pet adjustment device 900 of the embodiment.

[0146] It should be understood that the electronic devices in this application may include smartphones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices (MIDs) or wearable devices. The above electronic devices are only examples and are not exhaustive, including but not limited to the above electronic devices. In actual applications, the above electronic devices may also include: smart car terminals, computer equipment, etc.

[0147] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement part or all of the steps of any virtual pet adjustment method described in the above method embodiments.

[0148] An embodiment of the present application further provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any virtual pet adjustment method described in the above method embodiments.

[0149] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0150] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0153] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0154] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0155] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0156] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A virtual pet adjustment method, characterized in that: The method is applied to a virtual pet adjustment system, which includes: a virtual pet adjustment device and a wearable device; the multiple wearable devices maintain a communication connection with the virtual pet adjustment device; the method includes: The virtual pet adjustment device receives the user's body parameters from the wearable device; The virtual pet adjustment device determines a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet, including: determining the user's emotion based on the physical parameter; The body parameters include a plurality of first parameter values ​​of a plurality of physiological parameters at each sampling moment and a second parameter value of a motion parameter at each sampling moment; Vectorizing the multiple first parameter values ​​of the multiple physiological parameters at each sampling moment to obtain multiple first eigenvectors at each sampling moment, wherein each physiological parameter corresponds to one first eigenvector at each sampling moment; All the first eigenvectors at multiple sampling moments are concatenated to obtain a first eigenmatrix; Vectorizing a second parameter value of the motion parameter at each sampling moment to obtain a second eigenvector at each sampling moment, wherein the dimensions of the plurality of first eigenvectors at each sampling moment and the second eigenvector at each sampling moment are the same; splicing the plurality of second eigenvectors at the plurality of sampling moments to obtain a second eigenmatrix; Determining a weight matrix based on the first feature matrix and the second feature matrix; Performing weighted processing on the first characteristic matrix according to the weight matrix to obtain a third characteristic matrix; Performing emotion recognition based on the third feature matrix to obtain the user's emotion; Determining a target appearance and target body shape that matches the virtual pet based on a correspondence between the user's emotion, the type, appearance, and body shape of the virtual pet; The virtual pet adjustment device adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

2. The method according to claim 1, characterized in that The determining of a weight matrix based on the first feature matrix and the second feature matrix includes: respectively determining the similarity between a first eigenvector of each physiological parameter in the first feature matrix at each sampling moment and a second eigenvector of each physiological parameter in the second feature matrix at each sampling moment; Normalizing the multiple similarities of the multiple physiological parameters at each sampling moment to obtain weight coefficients of the multiple physiological parameters at each sampling moment; The weight coefficients of the multiple physiological parameters at each sampling moment are longitudinally replicated and combined to obtain the weight matrix.

3. The method according to claim 1 or 2, characterized in that After determining the user's emotion based on the body parameters, the method further includes: determining voice parameters corresponding to the virtual pet based on the user's emotions; Acquiring conversation content corresponding to the user based on the user's emotion; Determining a reminder time according to the user's emotion; At the reminder moment, the virtual pet is controlled to have the conversation with the user through the voice parameters.

4. The method according to claim 3, characterized in that Before the virtual pet adjustment device receives the user's body parameters from the wearable device, the method further includes: When it is detected that the current moment is within a preset time period, the virtual pet adjustment device sends a first instruction to the wearable device, wherein the first instruction is used to instruct the wearable device to report the user's location information to the virtual pet adjustment device; The virtual pet adjustment apparatus receives a response message to the first instruction from the wearable device, wherein the response message includes the location information of the user; When it is detected that the position information is within a preset range, the virtual pet adjustment device sends a second instruction to the wearable device, where the second instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment device.

5. The method according to claim 4, characterized in that Before the virtual pet adjustment device receives the user's body parameters from the wearable device, the method further includes: The virtual pet adjustment device obtains a start-up request from the user for the virtual pet; In response to the start-up request, the virtual pet adjustment apparatus sends a third instruction to the wearable device, where the third instruction is used to instruct the wearable device to report the user's physical parameters to the virtual pet adjustment apparatus.

6. A virtual pet adjustment device, characterized in that: The virtual pet adjustment device is used to implement the virtual pet adjustment method according to any one of claims 1 to 5, and the virtual pet adjustment device includes: a transceiver unit and a processing unit; The transceiver unit is configured to receive the user's body parameters from the wearable device; The processing unit is configured to determine a target appearance and a target body shape that match the virtual pet according to the body parameters and the type of the user's virtual pet, including: determining the user's emotion based on the physical parameter; The body parameters include a plurality of first parameter values ​​of a plurality of physiological parameters at each sampling moment and a second parameter value of a motion parameter at each sampling moment; Vectorizing the multiple first parameter values ​​of the multiple physiological parameters at each sampling moment to obtain multiple first eigenvectors at each sampling moment, wherein each physiological parameter corresponds to one first eigenvector at each sampling moment; All the first eigenvectors at multiple sampling moments are concatenated to obtain a first eigenmatrix; Vectorizing a second parameter value of the motion parameter at each sampling moment to obtain a second eigenvector at each sampling moment, wherein the dimensions of the plurality of first eigenvectors at each sampling moment and the second eigenvector at each sampling moment are the same; splicing the plurality of second eigenvectors at the plurality of sampling moments to obtain a second eigenmatrix; Determining a weight matrix based on the first feature matrix and the second feature matrix; Performing weighted processing on the first characteristic matrix according to the weight matrix to obtain a third characteristic matrix; Performing emotion recognition based on the third feature matrix to obtain the user's emotion; Determining a target appearance and target body shape that matches the virtual pet based on a correspondence between the user's emotion, the type, appearance, and body shape of the virtual pet; The virtual pet adjustment device adjusts the appearance of the virtual pet to the target appearance, and adjusts the body shape of the virtual pet to the target body shape.

7. An electronic device, characterized in that: include: A processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 5.

Images