A pet online interaction system operation method

Abstract

The application discloses a pet online interaction system operation method, collects real-time audio data and video data; uses a smell sensor to collect the smell of a pet, converts the collected smell information into an electric signal, and transmits the electric signal to a remote server through a network; monitors whether the pet meets the conditions of online interaction according to the collected real-time audio data, video data and smell; when it is monitored that the pet meets the conditions of online interaction, a first interactive pet is determined from the current pets that are online and in an idle state by using a preset pet recommendation method, and the target pet and the interactive pet are caused to interact through an interaction box; and a communication link is established between the target pet and the interactive pet. Through remote transmission of the smell of the pet, the pet owner can more deeply understand the character and characteristics of the interactive pet, and select a more suitable playmate or partner for the target pet; the interaction box is a relatively closed environment, which helps the target pet focus on interaction with the interactive pet and avoids external interference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote interaction, in particular to a pet online interaction system running method. BACKGROUND

[0002] With the improvement of living standards, more and more people keep pets. Because many pet owners need to go out to work, pets have to stay alone at home for a long time. For pets living in urban apartments, they may even have no chance to go out of the house without the company of their owners. Over time, the physical and mental health of pets may be affected. Based on the rapid development of technology and the continuous improvement of people's love for pets, the pet online interaction system has gradually become a new way of communication between pet owners and pets. Traditional pet feeding methods are often limited to face-to-face interaction, but the fast pace of modern life and heavy work pressure make it difficult for pet owners to be with their pets at all times.

[0003] For example, Chinese Patent No. CN114821815B discloses a pet online interaction system running method, device, equipment and medium, which comprises: acquiring real-time audio and video data sent by the host of the device end of the target pet, and monitoring whether the target pet in the video picture corresponding to the real-time audio and video data meets the preset online interaction condition by using a preset pet monitoring algorithm; if it is monitored that the target pet meets the preset online interaction condition, a first interaction pet is determined from the current online and idle pets by using a preset pet recommendation method; a first communication link is established between the device end of the target pet and the device end of the first interaction pet, so that the target pet and the first interaction pet can carry out online interaction. This method can recommend interaction friends for pets by analyzing pet sounds and movements, so that pets and pets can interact online, solve the problem of pet loneliness when there is no owner to accompany, and enrich the digital life of pets.

[0004] In the comparative document, there is no collection of pet odor, which is one of the important ways of pet communication. By restoring the odor of the pet, the pet owner can more intuitively understand the characteristics and personality of the pet, so that the selection of the first interaction pet is more accurate. SUMMARY

[0005] The present application provides a pet online interaction system running method, which collects the odor information of the interaction pet, so that the pet owner can obtain multi-dimensional information of the interaction pet, and the pet owner can more easily select the interaction pet for the target pet.

[0006] The present application provides a pet online interaction system running method, which comprises:

[0007] S101, collecting real-time audio and video data, acquiring real-time audio and video data of the pet;

[0008] Acquire real-time audio and video from the host of the pet device, and the host of the pet device collects real-time audio and video data through an integrated audio collection module and a video collection module;

[0009] S102, collect the smell of the pet using the smell sensor, convert the collected smell information into an electrical signal, and transmit it to the remote server through the network;

[0010] S103, monitor whether the pet meets the online interaction condition according to the collected real-time audio data, video data and smell;

[0011] S104, according to step S103, monitor that the target pet meets the online interaction condition, determine the first interactive pet from the current online and idle interactive pets using a preset pet recommendation method, and make the target pet interact with the interactive pet through the interactive box; the pet recommendation method constructs a weighted evaluation function according to the voice matching degree, the action matching degree, the user preference and the popularity;

[0012] S105, establish a communication link between the target pet and the first interactive pet;

[0013] S106, record the interactive action information and interactive voice information of the target pet and the first interactive pet in real time during the online interaction.

[0014] Preferably, the condition of whether the target pet meets the online interaction is monitored, if the complete body of the target pet in the video is in the field of view, or more than half of the body of the target pet is in the field of view and the head is towards the local video collector of the device end, it is determined that the condition is met, otherwise, it is determined that the condition is not met; if the concentration of the smell of the target pet exceeds the preset threshold, it is determined that the condition is met, if the concentration of the smell of the target pet does not exceed the threshold, it is determined that the condition is not met.

[0015] Preferably, the weighted evaluation function is w1 wherein w1, , and are weight coefficients.

[0016] Preferably, the method for making the target pet interact with the interactive pet through the interactive box is as follows:

[0017] S201, set the interactive box, the interactive box is provided with a main screen, an auxiliary screen, a smell collection module and a smell restoration module, the smell restoration module includes a smell generator and a release port;

[0018] The main screen is located at the center of the front of the interactive box body, the auxiliary screens are distributed around the main screen or on the sides of the box body, the odor release ports are located below the auxiliary screens, and the odor release ports are electrically connected with the auxiliary screens, each odor release port corresponds to an auxiliary screen;

[0019] S202, placing the interactive box around the pet, the odor collecting module collects the odor information of the pet and uploads it to the remote server, and the remote server generates a recommendation list after receiving the uploaded odor information;

[0020] The collected odor information is converted into an electrical signal and transmitted in real time to the remote server through the network;

[0021] S203, the odor restoration module restores the odor of the interactive pet according to the received odor information of the interactive pet through the built-in odor generator.

[0022] Preferably, the target pet interacts with the interactive pet through the interactive box, records the interaction data, generates a virtual pet according to the interaction data, and virtually interacts with the virtual pet when the interactive pet is not online.

[0023] Preferably, the step of virtual interaction of the target pet is:

[0024] S301, real-time collection of interaction data of the target pet and the interactive pet, extraction of interaction preferences and interaction habits of the target pet according to the interaction data;

[0025] S302, generating a virtual interactive pet according to the interaction process of the target pet and the interactive pet recorded in step S301;

[0026] S303, virtual interaction according to the virtual interactive pet generated in step S302;

[0027] S304, collecting interaction data of the target pet and the virtual interactive pet, and analyzing the preferences of the target pet through the collected interaction data;

[0028] S305, according to the preferences of the target pet analyzed in step S304, the system recommends a matched interactive pet.

[0029] Preferably, the basic information of the preferences of the target pet collected is converted into a numerical feature vector, the personality characteristics are converted into a numerical vector by using one-hot encoding, a weight is allocated to each feature, a weighted score is calculated according to the weight allocation, and an interactive pet with a weighted score greater than a preset recommendation threshold is recommended as a recommended interactive pet.

[0030] Preferably,

[0031] S401, according to the target pet interaction data collected in step S301, feature extraction is performed on the collected data;

[0032] S402, according to the features extracted in step S401, the state of the target pet is identified;

[0033] S403, according to the state of the target pet identified in step S402, the video clip is edited;

[0034] S404, according to the real-time state of the target pet, information is matched from the virtual interaction information library to guide the target pet.

[0035] Preferably, the feature extraction includes sound features, image features and smell features. For sound feature extraction, the system converts the sound signal from time domain to frequency domain through fast Fourier transform, extracts the pitch feature, and extracts the volume feature by calculating the peak value. For image feature extraction, the target pet face image is trained using a convolutional neural network. The posture feature of the target pet body is extracted by a key point detection algorithm, and the motion speed of the target pet in the video can be calculated using a feature point tracking algorithm. For smell feature extraction, the type of smell is identified by using a sensor array.

[0036] Preferably, the video is composed of a series of continuous frames. When the system detects a change in the state of the target pet, the starting time point of the change is recorded. The starting time point is the time corresponding to the frame at which the state begins to change. When the state of the target pet ends or changes to another state, the ending time point of the change is recorded. The ending time point marks the end of the current state. The recorded starting and ending time points are corresponding to the frame numbers of the video. Each frame has a unique number, so that the system can locate the time points to specific frames.

[0037] The one or more technical solutions provided in the application have at least the following technical effects or advantages:

[0038] By remotely transmitting the smell of the pet, the pet owner can better understand the personality and characteristics of the interactive pet, and choose a more suitable playmate or partner for the target pet. The interactive box is a relatively closed environment, which helps the target pet focus on interaction with the interactive pet and avoids external interference. The smell collection and restoration module equipped on the interactive box can accurately capture and restore the smell of the pet, enabling the pets to communicate more naturally and enhancing the realism of the interaction.

[0039] In addition, the system can also generate virtual interactive pet information according to the preferences and interaction quality of the target pet, and provide a realistic virtual interactive experience. Through comprehensive analysis of the target pet interaction records, the system can intelligently recommend more suitable interactive pets and improve the success rate of making friends. Even if the interactive pet is not online, the system can still provide a virtual interactive experience.

[0040] The system not only comprehensively records the interaction process of the target pet, but also identifies different states of the target pet through data analysis, automatically edits video clips, and provides personalized video sharing content for pet owners. Through matching and playing of virtual interaction information, the system can realize intelligent target pet state guidance, so that pet owners can check the target pet state, share videos and guide the state at any time through mobile devices such as mobile phones. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a pet online interaction system operation method of the present application;

[0042] Figure 2 A structural schematic diagram of an interactive box of an embodiment of the present application;

[0043] Figure 3 A flowchart of an interaction between a target pet and an interactive pet of an embodiment of the present application;

[0044] Figure 4 A flowchart of a target pet virtual interaction of an embodiment of the present application;

[0045] Figure 5 A flowchart of guiding the state of a target pet of an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings; the preferred embodiments of the present application are shown in the drawings, but the present application can be realized in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0047] It should be noted that the terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are only for the purpose of illustration and do not represent the only embodiment.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application; and the use herein of the terms "and / or" includes a combination of one or more of the associated listed items, in any of their possible permutations.

[0049] Embodiment one: Figure 1 is a flowchart of a pet online interaction system running method according to an embodiment of the present application, comprising the following steps:

[0050] S101, collect real-time audio and video data, and obtain real-time audio and video data of the pet.

[0051] Specifically, real-time audio and video are obtained from the host of the pet device, and the host of the pet device collects real-time audio and video data through an integrated audio collection module and a video collection module. The audio collection module is composed of a high-sensitivity microphone, which can capture various sounds emitted by the pet, such as barking and purring. The microphone converts the captured sound signals into electrical signals and processes them digitally through an audio codec, finally generating high-quality audio data. The generated audio data is transmitted to the host in real time for processing and analysis. The video collection module is composed of a high-definition camera, which can capture real-time activity pictures of the pet. The camera captures the image information of the pet through the lens and converts it into an electrical signal through an image sensor. The converted electrical signal is processed by an image codec to generate high-quality video data. The video data is also transmitted to the host in real-time for processing and analysis, so as to identify different states of the pet and clip personalized video segments.

[0052] S102, use the smell sensor to collect the smell of the pet, convert the collected smell information into an electrical signal, and transmit it to the remote server through the network.

[0053] Further, the pet includes a target pet and an interactive pet, and the smell sensor used is an electronic nose, which is a device that can simulate the biological olfactory function and can identify and quantify a variety of odor molecules. The electronic nose is composed of a sensor array, a signal processing unit and a data transmission module. The electronic nose is placed in the area where the pet is active to collect the smell of the pet. The sensor array of the electronic nose collects odor molecules in the air. The collected molecules react with the surface of the sensor to generate a measurable electrical signal. An analog-to-digital converter converts the analog electrical signal generated by the electronic nose into a digital signal. The converted digital signal is encoded for network transmission and storage. The encoding process may include data compression, encryption and other steps to reduce bandwidth requirements and improve data security. The electronic nose device is connected to a remote server through a network. The encoded smell data is transmitted to the remote server through the network. The remote server receives the smell data transmitted from the electronic nose device and performs preliminary processing. The processed smell data is stored in the database on the server side.

[0054] S103, according to the collected real-time audio data, video data and smell data, whether the pet meets the online interaction condition is monitored.

[0055] The video screen corresponding to the real-time audio and video data and the smell concentration of the pet are monitored to determine whether they meet the preset online interaction condition. If the complete body of the pet is in the field of view, or more than half of the body of the pet is in the field of view and the head is facing the local video collector of the device, it is determined that the condition is met. Otherwise, it is determined that the condition is not met. If the concentration of the smell of the pet exceeds the preset threshold, it is determined that the condition is met. If the concentration of the smell of the pet does not exceed the threshold, it is determined that the condition is not met. The threshold is set according to historical data and professional experience.

[0056] S104, according to the monitoring in step S103 that the pet meets the online interaction condition, a first interactive pet is determined from the interactive pets that are currently online and in an idle state by using a preset pet recommendation method, and the target pet and the interactive pet are made to interact through the interactive box.

[0057] The system determines a set of voice words and a set of action words for the pet. When the pet interacts, the system collects voice samples of the pet in real time, which contain various sounds emitted by the pet in different situations and moods. The system extracts Mel Frequency Cepstral Coefficients (MFCC) from the voice samples using signal processing techniques. MFCC is a common voice feature that can effectively represent the spectral envelope information of the voice. Based on the extracted MFCC features, a corresponding voice feature vector is generated for each voice sample. The system performs clustering analysis on all voice feature vectors using a preset clustering algorithm (DBSCAN), classifies similar voice feature vectors into a class, and generates voice clustering vectors. Each voice clustering vector is a voice word, and several voice words constitute the voice dictionary of the pet. The system collects video information in real time, which contains the dynamic performance of the pet in various activities. The system extracts optical flow information and color information of each frame of image from the video information. The optical flow information reflects the motion trajectory and speed of the pet, while the color information helps to identify the appearance characteristics of the pet. The extracted optical flow and color information are input into a preset deep neural network. After training by the deep neural network, the feature vector representing the action features of the pet can be output. The system performs clustering analysis on the action feature vectors using a preset clustering algorithm, generates a preset number of action clustering vectors, and each action clustering vector is an action word. Multiple action words constitute the action dictionary of the pet.

[0058] The system calculates the matching degree of the voice word set of the target pet and the voice ability of the potential interactive pet, using the method of cosine similarity. The system calculates the matching degree of the action word set of the target pet and the behavior ability of the potential interactive pet. Based on the pet owner's past preferences for the pet, such as favorite pet type, color, size, etc., the preference information is obtained by analyzing the user's historical interaction records. Based on the popularity of the pet, such as the number of interactions of the pet, user evaluation, etc., the popularity of the pet reflects the general appeal and interaction quality of the pet. According to the voice matching degree, the action matching degree, the user preference and the popularity, a weighted evaluation function is constructed, and the weighted evaluation function is w1 , and is a weight coefficient. According to the calculated weighted evaluation function, the pet with the highest score is selected as the first interactive pet.

[0059] S105, a communication link is established between the target pet and the first interactive pet.

[0060] ​The device end of the target pet initiates a connection request to the device end of the first interactive pet, the device end of the first interactive pet verifies the validity and security of the request after receiving the connection request, and after verification, the device end of the first interactive pet accepts the connection request, and after the target pet device end and the first interactive pet device end confirm the connection, the system will use a network communication protocol (TCP / IP,) to establish a stable communication link.

[0061] S106, record the interactive action information and interactive voice information of the target pet and the first interactive pet in real time during online interaction.

[0062] The technical solutions in the embodiments of the application have at least the following technical effects or advantages:

[0063] By remotely transmitting the smell of the pet, the pet owner can obtain more information about the interactive pet, so as to select the first interactive pet. By transmitting the smell of the interactive pet, the pet owner can more intuitively understand the characteristics and personality of the interactive pet, and can quickly select the first interactive pet. Suitable interactive pets have a positive impact on the psychological and physical health of pets. With the aid of smell information, the pet owner can more easily find a suitable interactive pet for the target pet.

[0064] Embodiment two: based on the transmission of the smell in embodiment one, an interactive box is provided in this embodiment, the interactive box is provided with a smell collecting module and a smell restoring module, the interactive box provides a physical isolation space to ensure that the pet will not be disturbed or harmed by external factors during the interaction.

[0065] As shown in Figure 2 and Figure 3 , the method for the target pet and the interactive pet to interact through the interactive box is as follows:

[0066] S201, set up an interactive box, the interactive box is provided with a main screen, an auxiliary screen, a smell collecting module and a smell restoring module, the smell restoring module includes a smell generator and a release port.

[0067] Further, the main screen is located at the center of the front of the interaction box, and adopts high-definition touch screen technology, which not only provides clear visual effect, but also allows the pet owner to conveniently browse and select information through touch operation. The main screen will display the basic information of the pet, such as name, age, breed, etc., and can also play the video and display the pictures of the pet, helping the pet owner to more comprehensively understand the state and living habits of the pet. The auxiliary screens are distributed around the main screen or on the side of the box, mainly displaying high-definition pictures of specific body parts of the pet. The pet owner can observe different parts of the pet in more detail, thereby improving the understanding and love for the pet. Each auxiliary screen corresponds to a specific body part, such as head, paw, body, etc., so that the information display is more intuitive and orderly. The smell releasing port is located below or beside the auxiliary screen, and is electrically connected between the smell releasing port and the auxiliary screen. Each releasing port corresponds to an auxiliary screen. When the screen displays the picture of a specific part, the corresponding smell releasing port will release the smell of that part. The design of the smell releasing port not only enhances the sensory experience of the interaction box, but also enables the pet owner to more realistically feel the presence and breath of the pet.

[0068] S202, the interaction box is placed near the pet, the smell collecting module collects the smell information of the pet, and uploads the smell information to the remote server. After receiving the uploaded smell information, the remote server generates a recommendation list.

[0069] The smell collecting module is built-in with a smell sensor, which is an electronic nose. The electronic nose can accurately collect the smell information of the interactive pet. The collected smell information will be converted into an electrical signal and transmitted to the remote server in real time through the network.

[0070] After receiving the uploaded smell information, the remote server generates a recommendation list according to the basic information of the pet (such as breed, age, gender, etc.) and the pet owner's social needs (such as finding a playmate of the same breed, finding a pet with a compatible personality, etc.).

[0071] S203, the smell restoring module restores the smell of the interactive pet according to the received smell information of the interactive pet through the built-in smell generator.

[0072] In the interaction box, the pet owner selects to view the picture of the tail part through the touch screen. The smell restoring module releases the smell of the tail part from the smell releasing port at the tail according to the collected smell, so that the pet owner can further understand the characteristics of the pet, such as the softness of the hair and the health condition of the skin.

[0073] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0074] The environment in the interaction box is relatively closed, which helps the target pet focus more on the interaction with the interactive pet and is not distracted by external sounds, smells or other interference factors, so as to more accurately capture and analyze the behavior and voice information of the pet.

[0075] The smell collection module and the smell restoration module arranged on the interaction box can accurately collect and restore the smell of the pet. In the closed space, the transmission of the smell is more direct and effective, which helps the pets to communicate more naturally through the smell, thereby enhancing the authenticity of the interaction.

[0076] In the third embodiment, the interaction box is arranged in the second embodiment, so that the target pet interacts with the interactive pet more conveniently, records the communication process of the pet, evaluates the favorite pet friend, records the process of interaction with the interactive pet, and forms virtual friend information, which can be used for virtual interaction when the interactive pet is offline.

[0077] As shown in Figure 4 the steps of the virtual interaction of the target pet are as follows:

[0078] S301, real-time collection of interaction data of the target pet and the interactive pet, extraction of interaction preferences and interaction habits of the target pet according to the interaction data.

[0079] The interaction data includes sound, image, smell and action data. The sound collection uses a high-sensitivity microphone array to capture the sound of the target pet in all directions, including barking, purring, barking and the like. The image collection uses a high-definition camera to take a full-body photo of the target pet in real time. The camera has night vision function to ensure that the image of the target pet can be clearly captured in dark light environment. The smell collection uses an electronic nose (smell sensor) device. The action collection uses an infrared sensor to record the action data of the target pet in real time. The action data should include the behaviors of the pet such as walking, jumping, wagging tail and licking.

[0080] S302, generating a virtual interactive pet according to the interaction process of the target pet and the interactive pet recorded in step S301.

[0081] According to the collected interaction information, the preferences of the target pet are analyzed, and a virtual interactive object is created according to the preferences of the target pet. The virtual interactive object is modeled using 3ds Max, the basic framework of the virtual character is constructed using geometric bodies, the basic model is refined using the carving tools of the software, the hair, skin texture and eyes of the virtual pet are refined, the surface of the 3D model is mapped to a two-dimensional plane for drawing of the map, and the model is rendered using a rendering engine to generate a virtual interactive object.

[0082] S303, according to the virtual interactive pet generated in step S302, virtual interaction is carried out.

[0083] When the interactive pet is not online, the system automatically plays the voice of the virtual friend through the interactive box according to the preset schedule or the activity state of the target pet, displays the image of the virtual interactive object through the main screen and the auxiliary screen on the interactive box, and releases the smell of the virtual interactive object through the smell release port on the interactive box when the target pet shows loneliness or anxiety.

[0084] S304, collecting the interaction data of the target pet and the virtual interactive pet, and analyzing the preferences of the target pet through the collected interaction data.

[0085] Collect the interaction data of the target pet and the virtual pet, which includes but is not limited to the behavior record of the target pet (such as playing, eating, resting, etc.), the type of interactive object (such as human, virtual pet, toy, etc.), the time, place and result of interaction (such as the reaction of the target pet, emotional change, etc.), and analyze the collected data through statistical analysis method. The frequency data of the target pet interacting with the virtual pet in different time periods is obtained by statistical analysis method. The average interaction time of the target pet with the virtual pet per day is 2 hours, and the standard deviation of the interaction time is 0.5 hours. It is shown that the interaction time of the target pet with the virtual pet has certain volatility.

[0086] S305, according to the preferences of the target pet analyzed in step S304, the matching interactive pet is recommended.

[0087] The basic information of the collected preferences of the target pet is converted into a numerical feature vector. The character feature is converted into a numerical vector by using one-hot encoding. The age, body shape and health condition are directly represented by numerical values. According to the expert experience and actual data results, the weight of each feature is allocated. The weight of the character feature is high. The numerical value of each feature is multiplied by the corresponding weight to obtain the weighted score. The cosine similarity is calculated. The cosine value of two feature vectors is calculated. The interactive pet with a cosine value greater than the preset recommendation threshold is recommended as the recommended interactive pet.

[0088] In some embodiments, the information of the target pet and the three candidate pets is converted into a numerical feature vector, the feature vector is weighted and scored, the cosine similarity is used to calculate the similarity between the feature vectors of the target pet and the three candidate pets, and the similarity scores are obtained: candidate pet A (0.65), candidate pet B (0.70), and candidate pet C (0.85). The target pet prefers to be active in the afternoon and evening, and the candidate pet with a lively personality and a preference for interaction in the afternoon and evening is preferentially recommended. The score is dynamically adjusted, and the adjusted score is candidate pet A (0.65, unchanged), candidate pet B (0.75, slightly improved), and candidate pet C (0.90, significantly improved). According to the weighted score, the similarity calculation result, and the dynamic adjustment factor, the final recommendation list is generated, and the final recommendation list is: candidate pet C (highest matching degree), candidate pet B (second), and candidate pet A (lowest).

[0089] The technical solutions in the embodiments of the present application have at least the following technical effects or advantages:

[0090] According to the preferences and interaction quality of the target pet, personalized virtual friend information is generated to provide realistic virtual interaction experience. By analyzing the interaction records of the target pet, more suitable friends are intelligently recommended to improve the success rate of the target pet's friendship. By comprehensively recording the communication process and interaction quality of the pet, the system can generate personalized virtual friend information and provide realistic virtual interaction experience when the interactive pet is not online.

[0091] Embodiment four: By recording the interaction process between the target pet and the interactive pet, the interaction process is edited into a video and sent to the pet owner. The pet owner can determine the state of the target pet according to the received video, and operate the system to release the image, sound, and smell of the virtual pet according to the state of the target pet, and guide the state of the target pet, as shown in Figure 5

[0092] S401, according to the pet interaction data collected in step S301, the collected data is feature extracted.

[0093] Specifically, for the extraction of sound features, the system converts the sound signal from time domain to frequency domain through fast Fourier transform (FFT), extracts the pitch feature, and extracts the volume feature by calculating the peak value; for the extraction of image features, a convolutional neural network (CNN) is used to train the pet face image, which can recognize the expressions of happy, sad, angry, and surprised of the pet, and the posture features of the pet body are extracted through a key point detection algorithm, and the motion speed of the pet in the video can be calculated using a feature point tracking algorithm; for the extraction of odor features, a sensor array is used to identify the type of odor.

[0094] ​S402, identify the state of the target pet according to the features extracted in step S401.

[0095] In the step, the state of the target pet is identified using a machine learning algorithm. The machine learning algorithm using a neural network is used to train the extracted features. In the training stage, known state data is introduced as a learning sample to optimize the neural network. The extracted features are input into the optimized neural network, and the state of the target pet is identified according to the neural network. When the neural network detects that the target pet emits high-pitched, fast-paced calls, and exhibits active body posture and fast movement speed, it may judge that the target pet is in an excited state. Conversely, if the target pet emits low-pitched, slow-paced calls, and exhibits relaxed body posture and slow movement speed, the system may judge that the target pet is in a calm state.

[0096] S403, according to the state of the target pet identified in step S402, clip the video segment.

[0097] Specifically, when the system detects a change in the state of the target pet, it records the starting time point of the change. The starting time point is the time corresponding to the frame at which the state begins to change. The video is composed of a series of consecutive frames. When the state of the target pet ends or changes to another state, the ending time point of the change is recorded. The ending time point marks the end of the current state. The recorded starting and ending time points are corresponding to the frame numbers of the video. Each frame has a unique number, so that the system can locate the time points to specific frames. At the same time, the time stamp of the start and end of each state is recorded. The time stamp is in seconds or milliseconds. The time stamp indicates the specific time point of the state change in the video. According to the marked time points, the segments corresponding to each state are extracted from the original video. The edited video is sorted in chronological order, so that the pet owner can view the state of the target pet in chronological order.

[0098] In some embodiments, the video of a pet cat playing in the house, the system identifies the following state changes through the neural network. The pet cat starts running from a stationary state. The starting frame of this state change is frame 150, and the time stamp is 00:00:02.500 (i.e. 2 seconds and 500 milliseconds from the start of the video). The pet cat stops and sniffs the ground after running for a while. The ending frame of this state change is frame 300, and the time stamp is 00:00:05.000 (i.e. 5 seconds from the start of the video). The pet cat starts running again. The starting frame of this state change is frame 301, and the time stamp is 00:00:05.001 (since the frame rate is high, the time stamp may only differ by 1 millisecond).

[0099] S404, according to the target pet real-time state, matching the corresponding information from the virtual interaction information library to guide the target pet.

[0100] According to the current state, personality, preferences and habits of the target pet, the current state of the target pet is extracted, the extracted feature vector is used to establish a matching model, the current state, personality, preferences and habits of the target pet are quantified as comparable indicators, which are used as one input of the matching model, the image, sound and smell information features in the virtual interaction information library are also quantified as comparable indicators, which are used as another input of the matching model, the similarity of the two input information is calculated using cosine similarity, and a matching threshold is set. When the calculated similarity is greater than the preset matching threshold, it is regarded as a matching result.

[0101] Further, the pet owner displays pictures, plays sounds and releases smells to the target pet according to the preset order and time interval through the interactive box system interface, and guides the target pet. The system feedbacks the real-time reaction and state change of the target pet to the pet owner in the form of a chart, so that the pet owner can understand the guiding effect. According to the real-time feedback, the pet owner can adjust the guiding strategy in time, such as changing the order of playing content, adjusting the volume or the concentration of smell, to optimize the guiding effect. The system generates a summary report of the target pet state guidance regularly, which can help the pet owner better understand the needs of the target pet.

[0102] The technical solutions in the embodiments of the present application have at least the following technical effects or advantages:

[0103] The system can comprehensively record the interaction process of the pet, identify different states of the target pet through data analysis, automatically edit video clips according to the state of the target pet, and provide personalized video sharing content for the pet owner. Through the matching and playing of virtual interaction information, intelligent pet state guidance is realized. The pet owner can check the target pet state, share video clips and guide the state at any time and anywhere through mobile devices such as mobile phones.

[0104] The combination of pet interaction records, video editing and virtual interaction technology provides a new and intelligent pet state management method for pet owners. Through comprehensive recording of the interaction process of the pet, identification of different states and editing of video clips, the system can help the pet owner better understand the behavior patterns and emotional states of the pet, and realize intelligent guidance of the pet state through matching and playing of virtual interaction information.

[0105] The above merely describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for operating an online pet interaction system, characterized in that, include: S101 collects real-time audio and video data to obtain real-time audio and video data of the pet; The system acquires real-time audio and video data from the host device on the pet device, which uses integrated audio and video acquisition modules to capture the data. S102 uses an odor sensor to collect the pet's scent, converts the collected scent information into an electrical signal, and transmits it to a remote server via a network; S103, based on the collected real-time audio data, video data, and odor data, monitor whether the pet meets the conditions for online interaction; wherein, the pet includes the target pet and the interactive pet; S104, based on step S103, if the target pet meets the online interaction conditions, a first interactive pet is determined from the currently online and idle interactive pets using a preset pet recommendation method, and the target pet interacts with the interactive pet through the interaction box; the pet recommendation method constructs a weighted evaluation function based on voice matching degree, action matching degree, user preference and popularity. S105, Establish a communication link between the target pet and the first interacting pet; S106, records in real time the interaction actions and voice information between the target pet and the first pet during the online interaction process; The target pet interacts with the interactive pet through the interaction box, records the interaction data, generates a virtual pet based on the interaction data, and performs virtual interaction with the virtual pet when the interactive pet is offline; The steps for virtual interaction with the target pet are as follows: S301 collects real-time interaction data between the target pet and the interacting pet, and extracts the target pet's interaction preferences and habits based on the interaction data; S302, Based on the interaction process between the target pet and the interactive pet recorded in step S301, generate a virtual interactive pet; S303, Perform virtual interaction based on the virtual interactive pet generated in step S302; S304, Collect interaction data between the target pet and the virtual interactive pet, and analyze the target pet's preferences through the collected interaction data; S305, Based on the target pet's preferences analyzed in step S304, the system recommends matching interactive pets; The basic information on the preferences of the target pets is collected and converted into numerical feature vectors. The personality features are converted into numerical vectors using one-hot encoding. Weights are assigned to each feature, and a weighted score is calculated based on the weight assignment. Interactive pets with a weighted score greater than a preset recommendation threshold are recommended as interactive pets.

2. The method for operating a pet online interactive system as described in claim 1, characterized in that, The system monitors whether the target pet meets the conditions for online interaction. If the entire body of the target pet is in the field of view in the video, or if more than half of the target pet's body is in the field of view and its head is facing the local video capture device, then the conditions are met; otherwise, the conditions are not met. If the concentration of the target pet's odor exceeds a preset threshold, the condition is deemed met; otherwise, the condition is deemed not met.

3. The method for operating a pet online interactive system as described in claim 1, characterized in that, The weighted evaluation function = w1 Among them, w1, , and It is the weighting coefficient.

4. The method for operating a pet online interactive system as described in claim 1, characterized in that, The method for enabling the target pet to interact with the interactive pet through the interaction box is as follows: S201, Set up an interactive box, which is equipped with a main screen, an auxiliary screen, an odor collection module and an odor restoration module. The odor restoration module includes an odor generator and a release port. The main screen is located at the center of the front of the interactive box, and auxiliary screens are distributed around the main screen or on the side of the box. The odor release port is located below the auxiliary screen and is electrically connected to the auxiliary screen. Each odor release port corresponds to one auxiliary screen. S202, the interactive box is placed around the pet, the scent collection module collects the pet's scent information and uploads it to the remote server. After receiving the uploaded scent information, the remote server generates a recommendation list. The collected odor information is converted into electrical signals and transmitted to a remote server in real time via the network; S203, the scent restoration module restores the scent of the interactive pet by using the built-in scent generator based on the received scent information of the interactive pet.

5. The method for operating a pet online interactive system as described in claim 1, characterized in that, S401, Based on the target pet interaction data collected in step S301, feature extraction is performed on the collected data; S402, Identify the state of the target pet based on the features extracted in step S401; S403, Based on the target pet status identified in step S402, edit video clips; S404: Based on the target pet's real-time status, information is matched from the virtual interaction information database to guide the target pet.

6. The method for operating a pet online interactive system as described in claim 5, characterized in that, The feature extraction includes sound features, image features, and odor features. For sound feature extraction, the system converts the sound signal from the time domain to the frequency domain using a fast Fourier transform to extract pitch features and extracts volume features by calculating peak values. For image feature extraction, a convolutional neural network is used to train the target pet's facial image. The target pet's body posture features are extracted using a keypoint detection algorithm, and the target pet's movement speed in the video can be calculated using a feature point tracking algorithm. For odor feature extraction, the type of odor is identified using a sensor array.

7. The method for operating a pet online interactive system as described in claim 6, characterized in that, The video consists of a series of consecutive frames. When the system detects a change in the target pet's state, it records the start time of the change. The start time is the time corresponding to the frame where the state begins to change. When the target pet's state ends or changes to another state, it records the end time of the change. The end time marks the end of the current state. The recorded start and end times are matched with the video frame numbers. Each frame has a unique number, allowing the system to pinpoint the time point to a specific frame.

Images