Virtual reality space video calling method and system for nursing home

Abstract

The invention discloses a virtual reality space video calling method and system for a nursing home. The virtual reality space video calling method for the nursing home comprises the following steps: acquiring head portrait information of a user; acquiring identity information of a user; a virtual human motion path provided by a user is taken, and the virtual human motion path comprises a plurality of movable areas needing to pass through; obtaining a memory condition; historical video information is called for each movable area according to a recall condition; and the data center sends the historical video information corresponding to the movable area to the control terminal when the virtual person passes through the movable area corresponding to the historical video information, so that the control terminal plays the historical video information. Historical videos in each movable area of a path are extracted through a virtual human motion path provided by a user, so that the user can see previous images in a familiar place, and old people can think previous life through the images.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and in particular to a virtual reality space interaction method for a nursing home and a virtual reality space interaction system for a nursing home. Background Art

[0002] The elderly people living in nursing homes are usually older, and usually live in couples or with a few friends, so that they can take care of each other and accompany each other. However, in life in nursing homes, there will always be situations where one of the couple dies first while the other is still healthy, or there may be situations where some elderly people are in poor health and cannot go out for a walk often.

[0003] In this case, if the elderly want to watch their past memories, the existing technology cannot satisfy them.

[0004] Therefore, it is hoped that a technical solution can be provided to solve or at least alleviate the above-mentioned deficiencies of the prior art. Summary of the invention

[0005] The purpose of the present invention is to provide a virtual reality space video calling method for a nursing home to solve at least one of the above-mentioned technical problems.

[0006] The present invention provides the following scheme:

[0007] According to one aspect of the present invention, a method for calling a virtual reality space video for a nursing home is provided, and the method for calling a virtual reality space video for a nursing home comprises:

[0008] The data center obtains the user's head portrait information through the onboard camera device of the control terminal;

[0009] The data center obtains the user's identity information based on the user's avatar information;

[0010] The data center obtains a virtual human motion path provided by a user, wherein the virtual human motion path includes a plurality of movable areas that need to be passed through;

[0011] Data center obtains recall conditions;

[0012] The data center retrieves historical video information for each active area based on the recall conditions;

[0013] When the virtual person passes through the movable area corresponding to the historical video information according to the movement path of the virtual person, the data center sends the historical video information corresponding to the movable area to the control terminal, so that the control terminal plays the historical video information.

[0014] Optionally, the data center obtains the recall condition including:

[0015] After obtaining the motion path of the virtual person, respectively obtaining the name of each movable area to be passed through in the motion path of the virtual person and the preset image of the movable area;

[0016] Sending the name of each movable area and the preset image of the movable area to the control terminal;

[0017] The control terminal displays the acquired name of the movable area and the preset image of the movable area and generates question information;

[0018] The control terminal obtains answer information fed back by the user according to the question information;

[0019] The control terminal obtains the recall condition according to the answer information and sends it to the data center.

[0020] Optionally, the data center retrieves historical video information for each movable area according to the recall condition, including:

[0021] Do the following for each active area:

[0022] Acquire a user private video library, wherein the user private video library includes at least one user private video and a plurality of tags, and one user private video is associated with a plurality of tags;

[0023] It is determined whether there is one and only one tag associated with a user's private video that meets the recall condition. If so, the user's private video of the movable area is obtained.

[0024] Optionally, retrieving historical video information for each movable area according to the recall condition in the conference condition sub-library further includes:

[0025] When multiple tags associated with a user's private videos meet the recall criteria,

[0026] A user private video that meets a first preset condition among the tags associated with each user private video is obtained as the user private video of the movable area.

[0027] Optionally, the recall condition includes one or more of a date condition, a climate condition, a time condition, and a person condition;

[0028] The tags include a date tag, a climate tag, a time tag, and the tags include at least one person tag.

[0029] Optionally, when the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, the data center retrieves historical video information for each movable area according to the recall condition, including:

[0030] Determine whether the current user has a preset conditional priority database based on the user identity information, and if so,

[0031] The user-preset condition priority database is used to retrieve historical video information for each active area.

[0032] Optionally, when the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, the data center retrieves historical video information for each movable area according to the recall condition further including:

[0033] Determine whether the current user has a preset condition priority database based on the user identity information, if not, then

[0034] The basic priority database is used to retrieve historical video information for each movable area, wherein the priority of the person tag is higher than the climate tag, the priority of the climate tag is higher than the date tag, and the priority of the date tag is higher than the time tag.

[0035] Optionally, the historical video information transmitted by the data center to the control terminal is compressed historical video information, and the compressed historical video information is obtained by the following method:

[0036] Get historical video compression models;

[0037] The historical video information is input into the historical video compression model to obtain compressed historical video information.

[0038] The present application also provides a virtual reality space video calling system for a nursing home, and the virtual reality space video calling system for a nursing home includes:

[0039] Control terminals, the number of which is multiple, the address and location information of each control terminal are known, each control terminal is provided with a human-computer interaction system and a photographing system, the human-computer interaction system is used for human-computer interaction with a user, and the photographing system is used for photographing images according to instructions;

[0040] A data center, wherein a virtual reality space is provided on the data center;

[0041] A camera device, wherein the number of the camera devices is multiple and distributed in various locations of the nursing home; wherein,

[0042] The control terminal, system server and camera device cooperate to implement the virtual reality space video calling system for nursing homes as described in any one of claims 1 to 9.

[0043] The virtual reality space video calling method for nursing homes in this application has the following advantages:

[0044] The historical videos in each movable area of ​​the virtual human movement path provided by the user are extracted, so that the user can see the previous images in a familiar place, so that the elderly can miss their previous life through these images. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of a virtual reality space interaction method for a nursing home in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 The method for calling the virtual reality space video for the nursing home shown includes:

[0048] Step 1: The data center obtains the user's head portrait information through the onboard camera device of the control terminal;

[0049] Step 2: The data center obtains the user's identity information based on the user's profile picture information;

[0050] Step 3: The data center obtains a virtual human motion path provided by the user, wherein the virtual human motion path includes a plurality of movable areas that need to be passed through;

[0051] Step 4: The data center obtains the recall condition;

[0052] Step 5: The data center retrieves historical video information for each active area according to the recall conditions;

[0053] Step 6: When the virtual human passes through the movable area corresponding to the historical video information according to the movement path of the virtual human, the data center sends the historical video information corresponding to the movable area to the control terminal, so that the control terminal plays the historical video information.

[0054] In this embodiment, the data center obtains the recall condition including:

[0055] After obtaining the motion path of the virtual person, respectively obtaining the name of each movable area to be passed through in the motion path of the virtual person and the preset image of the movable area;

[0056] Sending the name of each movable area and the preset image of the movable area to the control terminal;

[0057] The control terminal displays the acquired name of the movable area and the preset image of the movable area and generates question information;

[0058] The control terminal obtains answer information fed back by the user according to the question information;

[0059] The control terminal obtains the recall condition according to the answer information and sends it to the data center.

[0060] In this embodiment, the data center retrieves historical video information for each movable area according to the recall condition, including:

[0061] For each active area, do the following:

[0062] Acquire a user private video library, wherein the user private video library includes at least one user private video and a plurality of tags, and one user private video is associated with a plurality of tags;

[0063] It is determined whether there is one and only one tag associated with a user's private video that meets the recall condition. If so, the user's private video of the movable area is obtained.

[0064] In this embodiment, retrieving historical video information for each movable area according to the recall conditions in the conference condition sub-library further includes:

[0065] When multiple user private videos are associated with tags that meet the recall criteria,

[0066] A user private video that meets a first preset condition among the tags associated with each user private video is obtained as the user private video of the movable area.

[0067] In this embodiment, the recall condition includes one or more of a date condition, a climate condition, a time condition, and a person condition;

[0068] The tags include a date tag, a climate tag, a time tag, and the tags include at least one person tag.

[0069] In this embodiment, when the recall condition includes a person tag, a date tag, a climate tag, and a time tag, the data center retrieves historical video information for each movable area according to the recall condition, including:

[0070] Determine whether the current user has a preset conditional priority database based on the user identity information, and if so,

[0071] The user-preset condition priority database is used to retrieve historical video information for each active area.

[0072] In this embodiment, when the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, retrieving historical video information for each movable area according to the recall condition in the conference condition sub-library further includes:

[0073] Determine whether the current user has a preset condition priority database based on the user identity information, if not, then

[0074] The basic priority database is used to retrieve historical video information for each movable area, wherein the priority of the person tag is higher than the climate tag, the priority of the climate tag is higher than the date tag, and the priority of the date tag is higher than the time tag.

[0075] In this embodiment, the data center obtains the virtual human motion path including:

[0076] Obtaining the user's identity information and obtaining the user's location information using a device for obtaining the user's head portrait information;

[0077] Obtain user demand information;

[0078] Analyze the user demand information, and determine whether it is necessary to establish a user virtual person according to the user demand information. If so,

[0079] Creating a user virtual person in the virtual reality space at the location of the user;

[0080] Acquiring task information according to the user demand information;

[0081] Generate a virtual human motion path based on character information.

[0082] The present application also provides a virtual reality space video calling system for a nursing home, and the virtual reality space video calling system for a nursing home includes:

[0083] Control terminals, the number of which is multiple, the address and location information of each control terminal are known, each control terminal is provided with a human-computer interaction system and a photographing system, the human-computer interaction system is used for human-computer interaction with a user, and the photographing system is used for photographing images according to instructions;

[0084] A data center, wherein a virtual reality space is provided on the data center;

[0085] A camera device, wherein the number of the camera devices is multiple and distributed in various locations of the nursing home; wherein,

[0086] The control terminal, system server and camera device cooperate to implement the virtual reality space video calling system for nursing homes as described above.

[0087] The present application also provides a virtual reality space interaction method for a nursing home, including:

[0088] Get user avatar information;

[0089] Determine whether the user's identity information can be obtained based on the user's profile picture information. If so,

[0090] Obtaining the user's identity information and obtaining the user's location information using a device for obtaining the user's head portrait information;

[0091] Obtain user demand information;

[0092] Analyze the user demand information, and determine whether it is necessary to establish a user virtual person according to the user demand information. If so,

[0093] Creating a user virtual person in the virtual reality space at the location of the user;

[0094] Acquiring task information according to the user demand information;

[0095] Controlling the user virtual person to complete the task information in the virtual reality space and generate a task report; wherein the virtual reality space includes twin spaces of various facilities in the nursing home;

[0096] Human-computer interaction is performed with the user according to the task report, thereby obtaining a new round of task information until the interaction ends.

[0097] The virtual reality space interaction method for nursing homes in this application has the following advantages:

[0098] (1) Considering that most of the users of nursing homes are elderly people, and elderly people are usually unlikely to control virtual people, this application only needs to obtain the user's demand information, and the system automatically controls the virtual person to complete the user's needs.

[0099] (2) The method of the present application enables the elderly to understand the specific conditions of each facility in the nursing home without leaving their homes, thereby preventing the elderly from going out without knowing the operating status of each facility, which results in wasting physical energy and time without being able to achieve the desired state. For example, if the restaurant is crowded or the restaurant's menu today does not have the elderly's favorite food, the method of the present application can directly obtain the number of people in the restaurant and the menu through a virtual person visiting the restaurant, so that the elderly can consider whether to go to the restaurant by themselves, without having to walk to the restaurant or call the restaurant to find out the situation.

[0100] In this embodiment, the user's head portrait information can be obtained by using a camera device provided on the control terminal. In this embodiment, the control terminal is a human-computer interaction device.

[0101] In this embodiment, judging whether the user identity information can be obtained according to the user's head portrait information includes:

[0102] Acquire a preset face database, wherein the preset face database includes at least one preset face image and user identity information corresponding to each preset face image;

[0103] The similarity between the user portrait information and each preset face image is calculated respectively. If the similarity between a preset face image and the user portrait information exceeds a preset threshold, it is judged as yes.

[0104] When the judgment is yes, the user identity information corresponding to the preset face image whose similarity exceeds the preset threshold is obtained.

[0105] In this embodiment, each camera device on the control terminal has its own ID number and preset location information. Therefore, the user's location information can be obtained by obtaining the preset location information of the camera device.

[0106] It is understandable that, since it is a nursing home, each elderly person's room will be equipped with a control terminal for the elderly to use.

[0107] In this embodiment, the user demand information can be provided in the form of voice or text. For example, a user can say in voice: I plan to go to the cafeteria to eat at 12 o'clock.

[0108] In one embodiment, parsing the user demand information and determining whether to establish a user virtual person according to the user demand information includes:

[0109] When the user provides the user demand information by voice, the voice is converted into text by voice recognition;

[0110] A communication preference database is obtained, wherein the communication preference database includes at least one preset identity and a personal text database corresponding to each preset identity; in this embodiment, each personal text database stores a large amount of keyword information, including various place name information in the nursing home (for example, canteen, cinema, it can be understood that since canteen, etc. may have other alternative nouns in real life, for example, the canteen of the nursing home is just a popular noun, and the actual noun may be Jiaochangchunyuan, etc., at this time, these nouns that can refer to the canteen are also all included in the place name information), and colloquial names frequently used by the user may also be set (for example, the canteen next to the cinema, at this time , it may refer specifically to a specific canteen, then the canteen near the cinema will be associated with the specific canteen, and when the text information appears about the canteen near the cinema, the canteen associated with it will be automatically identified), and also includes various noun information that can reflect the purpose in the nursing home (for example, eating, exercising, fitness, swimming, etc.), and also the colloquial noun information that can reflect the purpose commonly used by users (for example, the user likes to play cards, just say playing cards, and the corresponding exercise area for playing cards may be a rehabilitation center for the elderly. At this time, playing cards is associated with the rehabilitation center for the elderly. Once the user appears with the noun playing cards, it is automatically identified as a rehabilitation center for the elderly).

[0111] Obtaining a personal text database corresponding to a preset identity that is identical to the user identity information;

[0112] Segment the user demand information (i.e. text information) into words to obtain each phrase;

[0113] Comparing each phrase with each keyword information in the personal text database for similarity, thereby obtaining keyword information with a similarity exceeding a preset threshold, and obtaining each keyword information with a similarity exceeding the preset threshold;

[0114] When the obtained keyword information whose similarities exceed the preset threshold includes various place name information in the nursing home, it is judged that it is necessary to establish a virtual person for the user. It can be understood that the present application does not require the elderly to go out at this time. Therefore, even if the corresponding action information is not obtained, a virtual person can actually be established, because in this case it will not cause unnecessary burden to the elderly. For example, the user's text information is canteen. Assuming there are only two words, at this time, although there is no action information about what to do, a virtual person can be established, because maybe the elderly just want to go to the canteen for a stroll through the virtual person. Therefore, in this case, a virtual person is also established for the user. It can be understood that establishing a user virtual person in the virtual space is a prior art and will not be repeated here. For example, virtual person login in the metaverse, login of virtual characters in games, etc.

[0115] It is understandable that if the number of place name information obtained is multiple, the elderly can be allowed to select one as keyword information through human-computer interaction (as described above, for example, the cafeteria next to the cinema actually has two place name information, but when the actual word segmentation is performed, the existing technology can identify the cinema part as the attributive of the cafeteria through word segmentation recognition, that is, it will not be divided into two words, and the overall word has a corresponding noun associated with it. Therefore, this description is actually still a place name information after recognition, that is, the cafeteria associated with it).

[0116] In one embodiment, obtaining task information according to the user demand information includes:

[0117] Acquire a movable destination library, wherein the movable destination library includes a plurality of destination names and location information corresponding to each destination name;

[0118] Acquire a destination name identical to the keyword information from the movable destination library according to the acquired keyword information;

[0119] It is determined whether the acquired keyword information contains action information. If so, the destination name and the action information are combined to form task information.

[0120] It is determined whether the acquired keyword information contains action information. If not, the destination name is used alone as task information.

[0121] The advantage of adopting this method is that creating a virtual person requires a certain amount of computing power and network. If a virtual person is created without being sure whether there are clear instructions, it will waste computing power and network.

[0122] In this embodiment, the present application analyzes the user demand information and determines whether it is necessary to establish a user virtual person based on the user demand information. As long as it is determined that text information can be obtained through voice information or it is determined that text information is obtained, it is determined that a virtual person needs to be established. That is to say, in this embodiment, after obtaining the user demand information, a virtual person is directly established and then the specific task is analyzed.

[0123] In this embodiment, acquiring task information according to the user demand information includes:

[0124] A user expression habit database is obtained according to the user identity information, and the expression habit database includes a position sub-library and a behavior sub-library, wherein the position sub-library includes at least one nursing home active area name group; the behavior sub-library includes at least one behavior name group, and one behavior name group is associated with at least one nursing home active area name group; in this embodiment, one nursing home active area name group corresponds to an active area in the nursing home. For example, the nursing home has a cinema A and a cinema B, then cinema A has a corresponding active area name group A, and cinema B has an active area name group B. Regardless of the active area name group A or the active area name group B, multiple active area names are included therein. The reason for having multiple active area names is that different users may have different descriptions of the active area. For example, the name set for cinema A in the nursing home itself is the first active cinema, but some elderly people may habitually call it the old cinema when using the name. At this time, in order to be able to identify the description of the elderly, multiple possible descriptions are placed in one active area name group.

[0125] In this embodiment, there are multiple user expression habit databases, and one user expression habit database corresponds to one user identity information. Therefore, the corresponding user expression habit database can be obtained according to the user identity information.

[0126] According to the user demand information, the activity area name group of the nursing home required by the user is obtained from the position sub-library, and the behavior name group required by the user is obtained from the behavior sub-library; specifically, the activity area name group and the behavior name group required can also be determined by obtaining the activity area name that is the same as the keyword in the user demand information through word segmentation, semantic recognition and similarity;

[0127] Determine whether the obtained nursing home active area name group and behavior name group are related. If so, use the obtained nursing home active area name group and behavior name group as task information.

[0128] In this embodiment, both the activity area name and the behavior name are required to proceed to the next step. If not, questions can be asked to the user through intelligent interaction to obtain the missing information. The advantage of this method is that it can remind the user through voice inquiries in case the user forgets to describe the missing part.

[0129] In this embodiment, controlling the user virtual person to complete the task information in the virtual reality space and generating a task report includes:

[0130] At least two virtual human movement paths are generated from the current location of the virtual human to the target movable area corresponding to the movable area name group according to the current location of the virtual human and the movable area name group; it can be understood that any two locations in a general nursing home (such as location H and location O) cannot have only one passable road. Therefore, in order to allow users to make their own choices, at least two virtual human movement paths are set for users to choose from.

[0131] The names of the movable areas passed by in each virtual human motion path are obtained; when the user is selecting a path, he may consider the distance and speed on the one hand, and the scenery or facilities passed by on the other hand. Therefore, the names of the movable areas passed by in each virtual human motion path are obtained, so that when the user is provided with a choice, the user can consider selecting a path according to the places passed by.

[0132] Sending the names of the movable areas passed by each virtual human motion path to the user;

[0133] Acquire the selection information fed back by the user; it is understandable that in order to reduce the user's operations, the feedback selection information can be obtained by voice or direct screen selection.

[0134] Selecting one of the virtual human movement paths according to the selection information fed back by the user, thereby controlling the virtual human to move according to the selected virtual human movement path until reaching the target movable area;

[0135] After the virtual person reaches the target movable area, intelligence data is collected from the target movable area to generate a mission report.

[0136] In this embodiment, the virtual reality space of the present application can be constructed based on digital twin technology or metaverse technology. In the three-dimensional scene constructed based on digital twin technology or metaverse technology, various equipment and facilities that are consistent with reality exist in the corresponding positions. It is understandable that in order to save computing power, it is not necessary to model all equipment and facilities. For example, only ticket gates need to be established in the theater, because going to the theater is mainly to understand the current theater's movie schedule, the next movie content, and the number of scheduled people, etc., and there is no need to fully establish the detailed models inside. It is understandable that if there is no lack of computing power, a 1:1 restoration can also be performed.

[0137] In this embodiment, when one of the virtual human motion paths is selected according to the selection information fed back by the user, the virtual human is controlled to move according to the selected virtual human motion path until it reaches the movable area name group, the virtual reality space interaction method for the nursing home further includes:

[0138] Acquire a user virtual reality space preference database, wherein the user virtual reality space preference database is provided with at least one preset user preference sub-library and preset user identity information corresponding to each preset user preference sub-library;

[0139] Acquire a preset user preference sub-library corresponding to the preset user identity information that is the same as the user identity information;

[0140] Determine whether there is a preset scenery style preference setting in the preset user preference sub-library, and if so,

[0141] In the process of controlling the virtual human to move according to the selected virtual human movement path until reaching the movable area name group, the scenery along the way is displayed to the user according to the scenery style preference setting along the way.

[0142] In this embodiment, since the application adopts the method of virtual human movement to reach the destination, the basic form is the same as the method of the character in the game reaching the destination by walking. Both need to walk through a predetermined route. At this time, the user can observe the scenery along the way through the scenery style preference setting. The reason for adopting this method is mainly to allow the user to enjoy the scenery along the way. It is understandable that if the user does not need it, the virtual person can also be transmitted to each virtual transmission point in the virtual reality space of this application through coordinate point transmission, and travel to the destination through the transmission point, and the transmission point is usually closer to the destination.

[0143] In this embodiment, the scenery style preference setting along the way includes a real scenery style, an imaginary scenery style, and a real scenery historical style; wherein, when the scenery style preference setting along the way is set to the real scenery style, in the process of controlling the virtual person to move according to the selected virtual person movement path until reaching the movable area name group, displaying the scenery along the way for the user according to the scenery style preference setting along the way includes:

[0144] Obtaining a nursing home camera device range database, wherein the nursing home camera device range database includes the ID of each camera device of the nursing home and the shooting range information of each camera device;

[0145] According to the current coordinate position of the virtual person, video information of a camera device capable of photographing the current coordinate position is retrieved in real time and transmitted to a control terminal for interacting with the user.

[0146] In this way, the current video images of different areas are provided to users in different areas. For example, there is an outdoor stadium in the nursing home. Then, the video stream information of the outdoor stadium at the current time can be obtained through the camera device and transmitted to the control terminal used by the user, so that the user can see the current situation of the outdoor stadium. The reason for this is that on the one hand, some elderly people who do not want to go out can watch the external scenery, and on the other hand, the elderly can also observe the flow of people in different areas and whether there are acquaintances, which can provide some motivation for the elderly and encourage them to go out for exercise. In addition, the elderly can also see the weather conditions outside through the video. For example, even if it is raining or snowing at the time, the size of the snow in different locations can be obtained through the video, so that the user can further reasonably arrange his next travel plan.

[0147] In this embodiment, when the scenery style preference along the way is set to the real scenery historical style, the method for calling the virtual reality space video for the nursing home includes:

[0148] The data center obtains the user's head portrait information through the onboard camera device of the control terminal;

[0149] The data center obtains the user's identity information based on the user's avatar information;

[0150] The data center obtains a virtual human motion path provided by a user, wherein the virtual human motion path includes a plurality of movable areas that need to be passed through;

[0151] Data center obtains recall conditions;

[0152] The data center retrieves historical video information for each active area based on the recall conditions;

[0153] When the virtual person passes through the movable area corresponding to the historical video information according to the movement path of the virtual person, the data center sends the historical video information corresponding to the movable area to the control terminal, so that the control terminal plays the historical video information.

[0154] In some cases, the user may want to see scenery of a season different from the current season. For example, it is currently winter, and the user wants to see scenes of spring or summer. Then, the scenes of this spring or summer can be retrieved according to the elderly's memory conditions for the elderly to watch. It is understandable that historical videos of more specific time points can also be retrieved according to the elderly's memory conditions. For example, if the elderly miss the scene of taking a walk with someone in a nursing home on a certain day, historical videos of these places on that day can be retrieved.

[0155] In this embodiment, the data center obtains the recall condition including:

[0156] After obtaining the motion path of the virtual person, respectively obtaining the name of each movable area to be passed through in the motion path of the virtual person and the preset image of the movable area;

[0157] Sending the name of each movable area and the preset image of the movable area to the control terminal;

[0158] The control terminal displays the acquired name of the movable area and the preset image of the movable area and generates question information;

[0159] The control terminal obtains answer information fed back by the user according to the question information;

[0160] The control terminal obtains the recall condition according to the answer information and sends it to the data center.

[0161] For example, the name of the movable area obtained by the control terminal is a stadium, and the preset image of the movable area is an image near the stadium. The elderly are informed of the situation of the area by displaying both, and the elderly are asked questions (for example, what time do you want to review the video data) to obtain the elderly's answer information;

[0162] Perform semantic recognition on the answer information to identify the key words in it, such as time information, person information, etc.

[0163] In this embodiment, each user is provided with a user's private video library, which stores the user's historical video information. This information may include videos uploaded by the user, and may also include video information taken by various cameras in the nursing home that the user specifically stores.

[0164] Each video information can be set with multiple labels, such as time labels such as spring and summer, or character labels such as Zhang San and Li Si, or all of them.

[0165] In this embodiment, the recall condition includes one or more of a date condition, a climate condition, a time condition, and a character condition. For example, the recall condition may be: if you need to watch a video with Zhang San in October 2024, then October 2024 is a time tag, and Zhang San is a character tag.

[0166] When only one tag associated with a user's private video meets the requirement, the user's private video is directly selected.

[0167] When tags associated with multiple user private videos all meet the recall condition, a user private video that meets the first preset condition is selected as the user private video of the active area.

[0168] In this embodiment, the first preset condition can be set by the user. For example, the user can set the private video that is closest to the current time among all the user private videos that meet the recall condition as the video that meets the condition, or other conditions can be set.

[0169] In another embodiment, the recall condition includes one or more of a date condition, a climate condition, a time condition, and a person condition;

[0170] The tags include a date tag, a climate tag, a time tag and at least one person tag.

[0171] When the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, the data center retrieves historical video information for each movable area according to the recall condition, including:

[0172] Determine whether the current user has a preset conditional priority database based on the user identity information, and if so,

[0173] The user-preset condition priority database is used to retrieve historical video information for each active area.

[0174] In this embodiment, the conditional priority database can be set according to user needs. For example, the user sets that when a person tag, date tag, climate tag, and time tag are simultaneously present, the person tag shall prevail. That is, all movable areas that can retrieve historical video information must first satisfy the requirement of having the task tag, and secondly, the date tag, etc. Through layer-by-layer screening, the only movable area that can retrieve historical video information can be determined.

[0175] It is understandable that if the user has not set it, the system default logic can be used, for example, using the basic priority database to retrieve historical video information for each active area, wherein the person tag in the basic priority database has a higher priority than the climate tag, the climate tag has a higher priority than the date tag, and the date tag has a higher priority than the time tag.

[0176] When the data center of the present application transmits video data to the control terminal, compressed historical video information is transmitted. The compressed historical video information is obtained by the following method:

[0177] Get historical video compression models;

[0178] The historical video information is input into the historical video compression model to obtain compressed historical video information.

[0179] Specifically, the model for video processing includes a spatiotemporal feature extractor, a classifier and a decoder, wherein the spatiotemporal feature extractor is used to extract spatial features and fuse spatiotemporal features of video data frames, the classifier is used to receive video spatial features and video spatiotemporal features as input, and obtain stage classification results by further fusing video spatial features and video spatiotemporal features, and the decoder is used to receive video spatiotemporal features as input, and decode the video spatiotemporal features back to video frame data.

[0180] In this embodiment, the video processing model is trained by the following method:

[0181] Get the video data training set;

[0182] The video processing model is trained by using the video data training set, thereby obtaining a trained video processing model;

[0183] In this embodiment, the video processing model in the data center and the video processing model on the control terminal are the same video processing model.

[0184] In this way, the control terminal can use the video processing model of the present application for decoding when decoding.

[0185] In this embodiment, the video processing model is trained by using a video data training set, so that the trained video processing model is obtained, including:

[0186] Initialize model parameters and training parameters for video processing;

[0187] Inputting the video data training set into the spatiotemporal feature extractor to obtain video spatial features and video spatiotemporal features;

[0188] Inputting the video spatial features and the video spatiotemporal features into a classifier to obtain a predicted video frame category result;

[0189] Obtaining a first classification loss function according to the predicted video frame category result;

[0190] Input the video spatiotemporal features into the decoder to obtain the reconstructed video frame;

[0191] Obtaining a second classification loss function according to the reconstructed video frame;

[0192] Obtain a multi-task joint loss function according to the first classification loss function and the second classification loss function;

[0193] Repeat the above steps until the iteration is completed or the loss value tends to be stable, then the training is completed, and the model parameters at the time of training completion are obtained.

[0194] In this embodiment, obtaining a first classification loss function according to the predicted video frame category result includes:

[0195] The video spatial features and video spatiotemporal features are input into the classifier. The Trans layer in the classifier uses a multi-head self-attention mechanism to further fuse the spatial and temporal features, and classifies the fused features through a multi-layer perceptron MLP to obtain the predicted video frame category results. The loss is calculated with the real labels of the video frames in the video data training set to minimize the difference between the classification results of the video frames and the real label values. The classification loss function is as follows:

[0196]

[0197] Among them, L class represents the classification loss value, y is the true label value of the video frame, is the predicted video frame classification result, n is the number of categories, the loss value is calculated by the loss function, and the network parameters are updated in the direction of reducing the loss value. The network parameters that need to be updated in this part are the classifier and spatiotemporal feature extractor.

[0198] In this embodiment, obtaining the second classification loss function according to the reconstructed video frame includes:

[0199] The loss of the reconstructed video frame and the real surgical video frame in the video data training set is calculated to minimize the difference between the reconstructed video frame and the real video frame. The classification loss function is as follows:

[0200]

[0201] Among them, L recon represents the reconstruction loss value, y is the real video frame in the video data training set, is the reconstructed video frame generated by the model, C is the number of video frame categories, the loss value is calculated by the loss function, and the network parameters are updated in the direction of reducing the loss value. The network parameters that need to be updated in this part include the spatiotemporal feature extractor and decoder.

[0202] In this embodiment, obtaining the multi-task joint loss function according to the first classification loss function and the second classification loss function includes:

[0203] The classification loss and reconstruction loss of the integrated stage are combined to form a multi-task joint loss function:

[0204] L total =αL class +βL recon

[0205] Among them, α and β are weight parameters used to balance the two losses. The entire model parameters, including the spatiotemporal feature extractor, classifier, and decoder, are optimized through back propagation.

[0206] In this embodiment, the video processing model of the present application can be used when compression is performed, the video processing model of the present application can also be used when decompression is performed, and the video processing model of the present application can also be used when real-time video information recognition is performed. Specifically, a spatiotemporal feature extractor is used for compression, a decoder is used for decompression, and a spatiotemporal feature extractor and a classifier are used for recognition. The classifier can have multiple classification labels, for example, a specific scene (such as a basketball stand), a specific person, etc.

[0207] The compression model of this application can reduce bandwidth usage, accelerate video transmission, and not reduce image quality.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calling a virtual reality space video for a nursing home, characterized in that: The method for calling a virtual reality space video for a nursing home comprises: The data center obtains the user's head portrait information through the onboard camera device of the control terminal; The data center obtains the user's identity information based on the user's avatar information; The data center obtains a virtual human motion path provided by a user, wherein the virtual human motion path includes a plurality of movable areas that need to be passed through; Data center obtains recall conditions; The data center retrieves historical video information for each active area based on the recall conditions; When the virtual person passes through the movable area corresponding to the historical video information according to the movement path of the virtual person, the data center sends the historical video information corresponding to the movable area to the control terminal, so that the control terminal plays the historical video information.

2. The method for calling a virtual reality space video for a nursing home as claimed in claim 1, characterized in that: The data center obtains the recall condition including: After obtaining the motion path of the virtual person, respectively obtaining the name of each movable area to be passed through in the motion path of the virtual person and the preset image of the movable area; Sending the name of each movable area and the preset image of the movable area to the control terminal; The control terminal displays the acquired name of the movable area and the preset image of the movable area and generates question information; The control terminal obtains answer information fed back by the user according to the question information; The control terminal obtains the recall condition according to the answer information and sends it to the data center.

3. The method for calling a virtual reality space video for a nursing home as claimed in claim 2, characterized in that: The data center retrieves historical video information for each movable area according to the recall condition, including: Do the following for each active area: Acquire a user private video library, wherein the user private video library includes at least one user private video and a plurality of tags, and one user private video is associated with a plurality of tags; It is determined whether there is one and only one tag associated with a user's private video that meets the recall condition. If so, the user's private video of the movable area is obtained.

4. The method for calling a virtual reality space video for a nursing home as claimed in claim 3, characterized in that: The retrieving historical video information for each movable area according to the recall conditions in the conference condition sub-library further includes: When multiple tags associated with a user's private videos meet the recall criteria, A user private video that meets a first preset condition among the tags associated with each user private video is obtained as the user private video of the movable area.

5. The method for calling a virtual reality space video for a nursing home as claimed in claim 3, characterized in that: The recall condition includes one or more of a date condition, a climate condition, a time condition, and a person condition; The tags include a date tag, a climate tag, a time tag and at least one person tag.

6. The method for calling a virtual reality space video for a nursing home as claimed in claim 5, characterized in that: When the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, the data center retrieves historical video information for each movable area according to the recall condition, including: Determine whether the current user has a preset conditional priority database based on the user identity information, and if so, The user-preset condition priority database is used to retrieve historical video information for each active area.

7. The method for calling a virtual reality space video for a nursing home as claimed in claim 6, characterized in that: When the recall condition includes a person tag, a date tag, a climate tag, and a time tag at the same time, the data center further retrieves historical video information for each movable area according to the recall condition: Determine whether the current user has a preset condition priority database based on the user identity information, if not, then The basic priority database is used to retrieve historical video information for each movable area, wherein the priority of the person tag is higher than the climate tag, the priority of the climate tag is higher than the date tag, and the priority of the date tag is higher than the time tag.

8. The method for calling a virtual reality space video for a nursing home as claimed in claim 7, characterized in that: The historical video information transmitted by the data center to the control terminal is compressed historical video information, and the compressed historical video information is obtained by the following method: Get historical video compression models; The historical video information is input into the historical video compression model to obtain compressed historical video information.

9. A virtual reality space video calling system for nursing homes, characterized in that: The virtual reality space video calling system for the nursing home includes: Control terminals, the number of which is multiple, the address and location information of each control terminal are known, each control terminal is provided with a human-computer interaction system and a photographing system, the human-computer interaction system is used for human-computer interaction with a user, and the photographing system is used for photographing images according to instructions; A data center, wherein a virtual reality space is provided on the data center; A camera device, wherein the number of the camera devices is multiple and distributed in various locations of the nursing home; wherein, The control terminal, system server and camera device cooperate to implement the virtual reality space video calling system for nursing homes as described in any one of claims 1 to 8.

Images