Method and system for capturing character appearance in virtual reality space

Abstract

The invention discloses a role appearance capturing method and system in a virtual reality space. The role appearance capturing method in the virtual reality space comprises the following steps: acquiring user login information; the data center obtains user position information according to the user login information; according to the position information of the user, acquiring image information containing the user and transmitted by a camera device capable of shooting the position; the image information containing the user is identified through the trained YOLOv8 model, so that a human-shaped target is obtained; when there is only one human-shaped target, identifying the human-shaped target so as to obtain dressing information of the current human-shaped target; and the data center generates clothes of roles in the virtual reality space according to the obtained dressing information. According to the invention, the information of the clothes worn by the user at present can be captured after the user logs in the virtual reality space, and the type and style of the clothes worn by the role in the virtual reality space can be realized according to the information of the clothes worn at present.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method for capturing the appearance of a character in a virtual reality space and a system for capturing the appearance of a character in a virtual reality space. Background Art

[0002] Existing nursing homes have established corresponding virtual reality spaces, that is, the entire nursing home is formed into a large digital twin space. The elderly can log in to the virtual reality space directly through their mobile phones or terminals located in different areas of the nursing home, so that they can stroll around different areas of the entire nursing home without leaving their homes, including communicating with other virtual people.

[0003] However, in the prior art, each virtual person corresponding to an elderly person can only use virtual clothing or original clothing, and cannot display clothing corresponding to what the elderly person actually wore on that day.

[0004] Therefore, it is hoped that a technical solution can be provided to solve or at least alleviate the above-mentioned deficiencies of the prior art. Summary of the invention

[0005] The purpose of the present invention is to provide a method for capturing the appearance of a character in a virtual reality space to solve at least one of the above-mentioned technical problems.

[0006] The present invention provides the following scheme: According to one aspect of the present invention, a method for capturing the appearance of a character in a virtual reality space is provided, the method comprising: The data center obtains the user's login information; The data center obtains the user's location information based on the user's login information; The data center obtains image information containing the user transmitted by a camera device capable of photographing the location according to the user location information; The data center uses the trained YOLOv8 model to identify the image information containing the user, thereby obtaining the human-shaped target; When there is only one human-shaped target, the data center identifies the human-shaped target, thereby obtaining the clothing information of the current human-shaped target; The data center generates clothing for the character in the virtual reality space based on the acquired clothing information.

[0007] Optionally, the user login information further includes the user identity; The method for capturing the appearance of a character in a virtual reality space further comprises: When there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features; The data center determines whether there is an identified humanoid target with the same identity as the user. If so, The data center identifies the humanoid target corresponding to the identity of the humanoid target having the same identity as the user, thereby obtaining the clothing information of the current humanoid target; The data center generates a character in the virtual reality space based on the acquired clothing information.

[0008] Optionally, when there is only one human-shaped target, the data center identifies the human-shaped target, thereby obtaining the current clothing information of the human-shaped target, including: Extracting a human-shaped target image within a bounding box identified by the trained YOLOv8 model; Recognize the human-shaped target image, so as to obtain joint point targets in the human-shaped target image; Acquire an upper body region and a lower body region of a human-shaped target in a human-shaped target image according to the acquired joint point targets; Obtain a trained upper body clothing depth model and a lower body clothing depth model; Inputting the upper body area into the upper body clothing depth model to obtain an upper body clothing recognition result; The lower body area is input into the lower body clothing depth model to obtain the lower body clothing recognition result; wherein, The upper body clothing recognition result and the lower body clothing recognition result constitute the clothing information of the human-shaped target.

[0009] Optionally, the data center generates a character in the virtual reality space according to the acquired clothing information, including: Acquire a user clothing database, wherein the user clothing database includes at least one clothing type data group, each clothing type data group includes at least one clothing triplet, and each clothing triplet includes a clothing type data, a clothing image data, and a clothing virtual reality data; According to the clothing type data in each triplet, a triplet corresponding to the clothing type data that is the same as the upper body clothing recognition result is selected, and each selected triplet is called a candidate triplet; Calculate the similarity between each clothing image data in the candidate triplet and the upper body region, so as to obtain the candidate triplet corresponding to the clothing image data with the highest similarity as the final selected triplet; Obtaining virtual reality data of clothes in the final selected triplet; The clothing of the character in the virtual reality space is generated based on the subsequent clothing virtual reality data.

[0010] Optionally, when there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features, including: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If so, The identity of each human-shaped target is identified by facial features.

[0011] Optionally, when there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features further includes: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If not, then Determine whether the facial features of at least one human-shaped target in the image information containing the user can be obtained, and if so, Performing face recognition on the acquired facial features, thereby obtaining the identity of each human-shaped target with acquired facial features; Obtain full body features for humanoid targets without facial features; The whole body features are identified to obtain the identity of each humanoid target without facial features.

[0012] Optionally, the method for capturing the appearance of a character in a virtual reality space further comprises: The data center determines whether there is an identified humanoid target with the same identity as the user. If not, then The data center generates prompt action information and sends it to the user's handheld terminal; After sending the prompt action information to the user's handheld terminal, the data center obtains video information within a preset time period through a camera device that can capture the user's location information; The data center identifies the motion trajectory of each human-shaped target in the video information, and obtains a human-shaped target with the same identity as the user according to the motion trajectory.

[0013] Optionally, the trained YOLOv8 model includes a full-factor enhanced dynamic convolutional layer, wherein the full-factor enhanced dynamic convolutional layer is located in the backbone network of the YOLOv8 model, replacing the convolutional layer of the original YOLOv8 model.

[0014] The present application also provides a system for capturing the appearance of a character in a virtual reality space, the system comprising a data center, a camera device and a user handheld terminal, the data center, the camera device and the user handheld terminal cooperate to implement the method for capturing the appearance of a character in a virtual reality space as described above.

[0015] The method for capturing the appearance of a character in a virtual reality space of the present application enables a user to capture information about the clothes currently worn by the user after logging into the virtual reality space, and realizes the type and style of clothes worn by the character (i.e., virtual person) in the virtual reality space based on the information about the clothes currently worn. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a method for capturing the appearance of a character in a virtual reality space in one embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 The method for capturing the appearance of a character in a virtual reality space shown includes: Step 1: The data center obtains the user's login information; Step 2: The data center obtains the user's location information based on the user's login information; Step 3: The data center obtains image information containing the user transmitted by a camera device capable of photographing the location according to the user location information; Step 4: The data center uses the trained YOLOv8 model to identify the image information containing the user, thereby obtaining the human-shaped target; Step 5: When there is only one human-shaped target, the data center identifies the human-shaped target, thereby obtaining the clothing information of the current human-shaped target; Step 6: The data center generates clothing for the character in the virtual reality space based on the acquired clothing information.

[0019] The method for capturing the appearance of a character in a virtual reality space of the present application enables a user to capture information about the clothes currently worn by the user after logging into the virtual reality space, and realizes the type and style of clothes worn by the character (i.e., virtual person) in the virtual reality space based on the information about the clothes currently worn.

[0020] In this embodiment, the user login information further includes the user identity.

[0021] In this embodiment, the method for capturing the appearance of a character in a virtual reality space further includes: When there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features; The data center determines whether there is an identified humanoid target with the same identity as the user. If so, The data center identifies the humanoid target corresponding to the identity of the humanoid target having the same identity as the user, thereby obtaining the clothing information of the current humanoid target; The data center generates a character in the virtual reality space based on the acquired clothing information.

[0022] In this embodiment, the user login information may include IP information. For example, if the user login information is logged in through a fixed terminal, it may include the IP information of the fixed terminal, and the location of the fixed terminal is known, thereby obtaining the user location information.

[0023] On the other hand, the user login information may also include GPS information. For example, when a user logs in via a mobile phone terminal, the GPS information of the mobile phone terminal at that time may be uploaded during login, thereby obtaining the user's location information.

[0024] Alternatively, the user login information may also directly input the user location information. For example, if the user is currently dining in the lobby of restaurant A, the restaurant information may be directly input, such as the lobby of restaurant A, to obtain the user location information.

[0025] In this embodiment, the data center of the present application obtains the image information containing the user transmitted by the camera device capable of photographing the location according to the user location information, including: Acquire a position coordinate database, wherein the position coordinate database includes a plurality of position coordinate information and camera device ID information corresponding to each position coordinate information; The corresponding camera device ID information is obtained according to the obtained user location information.

[0026] In this embodiment, one piece of position coordinate information may correspond to multiple pieces of camera ID information, that is, multiple cameras may be able to capture the position coordinate information.

[0027] In this embodiment, no matter how the user location information is obtained (terminal IP information, GPS information or other methods), the location coordinate information can be obtained eventually. For example, after the terminal IP information is obtained, the pre-stored coordinate information of the terminal can be obtained as the user location information.

[0028] If it is in text form, such as information input by a user, each coordinate in a coordinate range can be used as user position information.

[0029] It is understandable that, since the present application requires logging into the virtual reality space, in theory, no matter what terminal the user logs in through, the corresponding GPS information can be obtained.

[0030] In this embodiment, the data center obtains the image information containing the user transmitted by the camera device capable of photographing the location according to the user location information, including: The data center can obtain the ID information of the camera device that can take the photo based on the user's location information and the above-mentioned location coordinate database, and thus generate a command to send to the corresponding camera device.

[0031] The camera device that receives the command sends the acquired video information (ie, multiple frames of images) to the data center.

[0032] It can be understood that the camera device that receives the command can send one frame of image information or multiple frames of image information for use by the data center. Regardless of whether it is one frame of image information or multiple frames of image information, when performing operations such as image recognition, the operation method is the same, that is, each frame of image information can be identified using the recognition method of the present application.

[0033] In this embodiment, when there is only one human-shaped target, the data center identifies the human-shaped target, and thus obtains the current clothing information of the human-shaped target, including: Extracting a human-shaped target image within a bounding box identified by the trained YOLOv8 model; Recognize the human-shaped target image, so as to obtain joint point targets in the human-shaped target image; Acquire an upper body region and a lower body region of a human-shaped target in a human-shaped target image according to the acquired joint point targets; Obtain a trained upper body clothing depth model and a lower body clothing depth model; Inputting the upper body area into the upper body clothing depth model to obtain an upper body clothing recognition result; The lower body area is input into the lower body clothing depth model to obtain the lower body clothing recognition result; wherein, The upper body clothing recognition result and the lower body clothing recognition result constitute the clothing information of the human target. In this embodiment, the clothing information may include the following information: sweatshirt, pajamas, short sleeves, long sleeves, shirts, etc.

[0034] In this embodiment, the data center generates a character in the virtual reality space according to the acquired clothing information, including: Acquire a user clothing database, wherein the user clothing database includes at least one clothing type data group, each clothing type data group includes at least one clothing triplet, each clothing triplet includes a clothing type data (such as hoodie, pajamas, short sleeve, long sleeve, shirt, etc.), a clothing image data and a clothing virtual reality data; According to the clothing type data in each triplet, a triplet corresponding to the clothing type data that is the same as the upper body clothing recognition result is selected, and each selected triplet is called a candidate triplet; that is, first find the recognition result that is the same as the clothing type data in the triplet by recognition. For example, if the recognition result is a shirt, then find the triplet whose clothing type data in each triplet is also a shirt.

[0035] The similarity between each clothing image data in the alternative triplet and the upper body area is calculated respectively, so as to obtain the alternative triplet corresponding to the clothing image data with the highest similarity as the final selected triplet; in this way, clothing virtual reality data of a shirt that is most similar to the triplet of each shirt can be obtained. It can be understood that each triplet of the present application can be input into the data center in advance.

[0036] Obtaining virtual reality data of clothes in the final selected triplet; The clothing of the character in the virtual reality space is generated based on the subsequent clothing virtual reality data.

[0037] In this embodiment, when there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features, including: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If so, The identity of each human-shaped target is identified by facial features.

[0038] It is understandable that when there are more than one humanoid targets, it is first necessary to determine which user is logging into the virtual space. Therefore, the identity of each humanoid target is first identified through feature recognition. It is understandable that identification can be performed through face recognition or full body feature recognition.

[0039] In this embodiment, when there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features, further comprising: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If not, then Determine whether the facial features of at least one human-shaped target in the image information containing the user can be obtained, and if so, Performing face recognition on the acquired facial features, thereby obtaining the identity of each human-shaped target with acquired facial features; Obtain full body features for humanoid targets without facial features; The whole body features are identified to obtain the identity of each humanoid target without facial features.

[0040] In this embodiment, the method for capturing the appearance of a character in a virtual reality space further comprises: The data center determines whether there is an identified humanoid target with the same identity as the user. If not, then The data center generates prompt action information and sends it to the user's handheld terminal; After sending the prompt action information to the user's handheld terminal, the data center obtains video information within a preset time period through a camera device that can capture the user's location information; The data center identifies the motion trajectory of each human-shaped target in the video information, and obtains a human-shaped target with the same identity as the user according to the motion trajectory.

[0041] It is understandable that if identity is recognized through facial features, it can usually be recognized. However, in some cases, facial features cannot be obtained. At this time, only full-body features (such as body features, clothing features, etc.) can be used for recognition. At this time, it may be possible that the identity cannot be recognized. At this time, the recognition can be assisted by generating prompt action information and requiring the user to actually operate. For example, the prompt action information is for the user to rotate a circle. In this case, it is only necessary to use the camera device to identify the human-shaped target that rotates a circle.

[0042] It is understandable that, whether using facial features or full body features, the identity of the humanoid target can be identified by comparing with pre-stored features through similarity calculation, which belongs to the prior art and will not be described in detail here.

[0043] In this embodiment, the trained YOLOv8 model used in this application includes a full-factor enhanced dynamic convolutional layer, wherein the full-factor enhanced dynamic convolutional layer is located in the backbone network of the YOLOv8 model, replacing the convolutional layer of the original YOLOv8 model.

[0044] Specifically, the structure of a Conv module of the backbone network of the existing YOLOv8 model is , we now replace it with a FEDC layer. Specifically, in the early stage of feature extraction, some representative convolutional layers can be selected for replacement to allow the model to learn richer multi-scale and contextual information. For example, in the shallow layer of the network, the 3rd to 5th convolutional layers (determined according to the actual model structure) can be replaced with FEDC layers, so that the model can capture complex features at an early stage.

[0045] Alternatively, the FEDC layer can be introduced in the PAN (Path Aggregation Network) and FPN (Feature Pyramid Network) parts. This part of the network is mainly responsible for feature fusion and transmission, and the multi-scale fusion characteristics of the FEDC layer can further enhance the feature fusion effect.

[0046] For example, in the upsampling and convolution operations of FPN, and in the downsampling and convolution operations of PAN, some key convolution layers can be replaced with FEDC layers. You can choose to replace the convolution layers that fuse feature maps of different scales, such as the convolution layers that transfer information from high-level feature maps to low-level feature maps, and the convolution layers that transfer information from low-level feature maps to high-level feature maps, which helps the model better utilize feature information at different levels.

[0047] In this embodiment, the FEDC layer (full factor enhanced dynamic convolution layer) of the present application includes the following modules: Ghost module, the input feature map first passes through the ghost module to generate multiple feature maps of different scales. The ghost module generates additional feature maps through a small number of convolution operations. These feature maps of different scales contain information of the input data at different resolutions, providing a basis for subsequent multi-scale feature fusion.

[0048] The weighted sum fusion module uses the attention mechanism function to calculate the weight coefficient of each scale feature map. The attention mechanism will comprehensively analyze all scale feature maps and assign weights according to the importance of the feature map to the current task. Then, the fused feature map is obtained by weighted summation.

[0049] A context information extraction module is used to process the fused feature map using a context encoder to extract context information from it. The context encoder is usually a complex neural network structure that can capture the relationship between the various parts in the feature map and their position and role in the entire scene. For example, in image tasks, it can help the model understand the relationship between different regions, so as to better identify the target object.

[0050] The dynamic convolution kernel generation module uses a function composed of a neural network to generate a dynamic convolution kernel with the fused feature map and context information as input. This function usually contains multiple convolution layers, fully connected layers, etc., and generates a convolution kernel suitable for the current input features by learning and processing the input information. The dynamic convolution kernel will change dynamically according to the different input features. Compared with the traditional fixed convolution kernel, it can adapt to various input modes more flexibly and enhance the expression ability of the model.

[0051] The normalization module is used to perform layer normalization on the fused feature map to obtain. Layer normalization is to normalize all features of each sample and adjust the mean and variance of the features to a specific range. This helps to stabilize the feature distribution, accelerate the convergence speed of model training, and improve the generalization ability of the model, so that the network can learn feature representation more effectively.

[0052] The parameter adjustment module is used to adopt a nonlinear activation function. First, the Swish activation function is used on the result of the convolution operation of the dynamic convolution kernel and the normalized feature map. The Swish activation function has the characteristics of smoothness and self-gating, which enables the model to learn richer feature representations. Then, the GELU activation function is used on the result after Swish activation. The GELU activation function can make the model better fit the data distribution and further improve the model's ability to learn complex nonlinear relationships.

[0053] Parameter adjustment: The results after nonlinear activation are adjusted through two learnable parameters. Together, they fine-tune the output results to optimize the performance of the model. Finally, after these operations, the output of the full-factor enhanced dynamic convolution layer is obtained.

[0054] In this embodiment, the specific formula of the FEDC layer of this application is as follows: ;in, Indicates the final output result; stands for Gaussian Error Linear Unit, which is an activation function that can adaptively adjust the output of neurons so that the model can better learn nonlinear relationships in the data; Represents a self-gated activation function, which is used to introduce nonlinear transformation and enhance the expressiveness of the model; The two parameters γ and β are similar to the scaling and translation parameters in Batch Normalization. γ is used to scale the values ​​processed by the activation function, and β is used for offset. Together, they can make more flexible adjustments to features.

[0055] It is a dynamic convolution kernel, which is dynamically generated according to the input data. It is used to perform convolution operations on the input feature map and extract features at different levels.

[0056] The bias term (Bias) of the dynamic convolution kernel provides a fixed offset for the convolution operation, which helps the model learn more complex functions.

[0057] Represents the input feature map after some normalization or preprocessing.

[0058] is a logical AND operation simulated by a step function.

[0059] It is the threshold in the logical AND operation, which is used to decide which features should be retained or ignored. It is an adjustable parameter. Different tasks and data sets may require different values ​​to optimize model performance.

[0060] The Exponent Parameter is used to control the degree of the power operation. The value can emphasize or suppress the feature to different degrees and is a learnable parameter.

[0061] Indicates taking the absolute value of the previous logic operation result and then performing a power operation. The value of can control the degree of feature emphasis or suppression to adapt to different data and task requirements.

[0062] Used to preserve the sign information of the previous operation result. This ensures that the positive and negative characteristics of the data are not lost during the calculation process, because the positive and negative signs may carry important semantic information in some cases.

[0063] is a very small positive number (usually around 10 -8 In the denominator middle, The function is to prevent the denominator from being 0 and ensure the stability of the normalization operation.

[0064] Represents a dynamic convolution kernel The i-th element of . Different i correspond to the values ​​of different positions of the convolution kernel, which determine the degree of weighting of the convolution kernel on different positions of the input data. For example, in a two-dimensional convolution kernel, elements at different positions perform weighted summation on pixels at different positions of the input feature map.

[0065] m is the upper limit of the number of convolution kernel elements, indicating the convolution kernel Accumulate the elements from the 1st to the mth element. For example, for a 3×3 two-dimensional convolution kernel, m may be 9. Here, the "strength" or "energy" of the convolution kernel is measured by performing subsequent operations on all elements of the convolution kernel.

[0066] By using the FEDC layer of this application to improve the existing YOLOv8 model, the model performance and learning ability can be significantly improved. The FEDC layer, relying on the design of dynamic convolution kernels, flexibly generates convolution kernels according to the input data. Compared with the traditional fixed convolution kernels, it can capture data features more accurately, greatly enhancing the adaptability of the model to complex and diverse data patterns. The logical operation part can effectively screen and highlight key features through logical and operations similar to step function simulation combined with threshold operations, while removing relatively minor information, which helps the model focus on important features for learning. Through the use of power operations and sign functions, the emphasis or suppression of features can be flexibly adjusted according to hyperparameters, and key sign information can be retained. This has a significant effect on feature processing with important semantics for directional or positive and negative characteristics, enriching the model's ability to express features. The normalization operation of the denominator ensures the stability of the calculation by reasonably calculating the sum of squares of the convolution kernel elements and adding a small constant to prevent division by zero, while making the features in different situations comparable, providing a guarantee for the stable conduct of model training and reasoning. The GELU function used in the activation function part, as a nonlinear activation function, can introduce strong nonlinearity into the neural network, allowing the model to learn extremely complex patterns and feature relationships in the data. Compared with traditional activation functions, it has advantages in convergence speed and task performance. In addition, the bias term in the formula provides a fixed offset for the convolution operation result, effectively reducing model deviation and assisting the model to better fit the data. In short, the multi-fusion dynamic convolution output formula comprehensively improves the performance, stability and ability to handle complex data of the model through the synergy of various parts, helping the model to achieve better performance in various tasks.

[0067] In one embodiment, the comprehensive loss function of the YOLOv8 model is:

[0068] in, Refers to classification loss, which measures the gap between the predicted category and the true category, using cross entropy loss; Position loss, which measures the gap between the predicted bounding box and the true bounding box; Refers to the pose estimation loss, which measures the gap between the predicted key point position and the true key point position; using the mean square error, These are the weight hyperparameters for the corresponding loss terms, which are used to balance the contribution of different loss terms to the total loss.

[0069] In this embodiment, the The classification loss calculation formula is as follows: ; Where N is the number of samples, C is the number of categories, It is a sample The true label belongs to category c, is the probability of class c predicted by the model.

[0070] In this embodiment, the The positioning loss calculation formula is as follows: ; in, It is a sample The ground-truth bounding box, is the bounding box predicted by the model.

[0071] In this embodiment, the The posture estimation loss calculation formula is as follows: ; Where K is the number of key points, It is a sample The true position of the jth key point, is the position of the jth keypoint predicted by the model.

[0072] The yolov8 model of this application is used to significantly improve the efficiency of feature extraction. It aims to reduce the computational complexity and the number of parameters by using partial convolution, thereby improving the reasoning speed and efficiency of the model, and can significantly reduce floating-point operations (FLOPs) and improve the reasoning speed.

[0073] The present application also provides a system for capturing the appearance of characters in a virtual reality space for a nursing home. The system for capturing the appearance of characters in a virtual reality space for a nursing home includes a data center, a camera device, and a user handheld terminal. The data center, the camera device, and the user handheld terminal cooperate to implement the method for capturing the appearance of characters in a virtual reality space as described above.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for capturing the appearance of a character in a virtual reality space, used in a nursing home, characterized in that: The method for capturing the appearance of a character in a virtual reality space comprises: The data center obtains the user's login information; The data center obtains the user's location information based on the user's login information; The data center obtains image information containing the user transmitted by a camera device capable of photographing the location according to the user location information; The data center uses the trained YOLOv8 model to identify the image information containing the user, thereby obtaining the human-shaped target; When there is only one human-shaped target, the data center identifies the human-shaped target, thereby obtaining the clothing information of the current human-shaped target; The data center generates clothing for the character in the virtual reality space based on the acquired clothing information.

2. The method for capturing the appearance of a character in a virtual reality space according to claim 1, characterized in that: The user login information further includes the user identity; The method for capturing the appearance of a character in a virtual reality space further comprises: When there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features; The data center determines whether there is an identified humanoid target with the same identity as the user. If so, The data center identifies the humanoid target corresponding to the identity of the humanoid target having the same identity as the user, thereby obtaining the clothing information of the current humanoid target; The data center generates a character in the virtual reality space based on the acquired clothing information.

3. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 2, characterized in that: When there is only one humanoid target, the data center identifies the humanoid target, thereby obtaining the current clothing information of the humanoid target, including: Extracting a human-shaped target image within a bounding box identified by the trained YOLOv8 model; Recognize the human-shaped target image, so as to obtain joint point targets in the human-shaped target image; Acquire an upper body region and a lower body region of a human-shaped target in a human-shaped target image according to the acquired joint point targets; Obtain a trained upper body clothing depth model and a lower body clothing depth model; Inputting the upper body area into the upper body clothing depth model to obtain an upper body clothing recognition result; The lower body area is input into the lower body clothing depth model to obtain the lower body clothing recognition result; wherein, The upper body clothing recognition result and the lower body clothing recognition result constitute the clothing information of the human-shaped target.

4. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 3, characterized in that: The data center generates a character in the virtual reality space according to the acquired clothing information, including: Acquire a user clothing database, wherein the user clothing database includes at least one clothing type data group, each clothing type data group includes at least one clothing triplet, and each clothing triplet includes a clothing type data, a clothing image data, and a clothing virtual reality data; According to the clothing type data in each triplet, a triplet corresponding to the clothing type data that is the same as the upper body clothing recognition result is selected, and each selected triplet is called a candidate triplet; Calculate the similarity between each clothing image data in the candidate triplet and the upper body region, so as to obtain the candidate triplet corresponding to the clothing image data with the highest similarity as the final selected triplet; Obtaining virtual reality data of clothes in the final selected triplet; The clothing of the character in the virtual reality space is generated based on the subsequent clothing virtual reality data.

5. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 4, characterized in that: When there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features, including: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If so, The identity of each human-shaped target is identified by facial features.

6. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 5, characterized in that: When there is more than one humanoid target, the data center captures the whole body features and / or facial features of each humanoid target respectively and identifies the identity of each humanoid target according to the whole body features and / or facial features, further comprising: Performing face detection on the image information containing the user, thereby obtaining facial features that can be obtained through the image information containing the user; Determine whether the facial features of all human targets in the image information containing the user can be obtained. If not, then Determine whether the facial features of at least one human-shaped target in the image information containing the user can be obtained, and if so, Performing face recognition on the acquired facial features, thereby obtaining the identity of each human-shaped target with acquired facial features; Obtain full body features for humanoid targets without facial features; The whole body features are identified to obtain the identity of each humanoid target without facial features.

7. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 6, characterized in that: The method for capturing the appearance of a character in a virtual reality space further comprises: The data center determines whether there is an identified humanoid target with the same identity as the user. If not, then The data center generates prompt action information and sends it to the user's handheld terminal; After sending the prompt action information to the user's handheld terminal, the data center obtains video information within a preset time period through a camera device that can capture the user's location information; The data center identifies the motion trajectory of each human-shaped target in the video information, and obtains a human-shaped target with the same identity as the user according to the motion trajectory.

8. The method for capturing the appearance of a character in a virtual reality space as claimed in claim 1, characterized in that: The trained YOLOv8 model includes a full-factor enhanced dynamic convolutional layer, wherein the full-factor enhanced dynamic convolutional layer is located in the backbone network of the YOLOv8 model, replacing the convolutional layer of the original YOLOv8 model.

9. A system for capturing the appearance of a character in a virtual reality space, characterized in that: The system for capturing the appearance of a character in a virtual reality space includes a data center, a camera device, and a user handheld terminal. The data center, the camera device, and the user handheld terminal cooperate to implement the method for capturing the appearance of a character in a virtual reality space as described in any one of claims 1 to 8.

Images