A virtual-reality interaction method and system based on image recognition technology

Abstract

The present invention relates to the field of virtual-reality interaction technology, specifically a virtual-reality interaction method and system based on image recognition technology, comprising: a target user graffitiing a drawing template with colored pens to obtain a colored drawing; performing three-dimensional modeling of an astronaut using three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the drawing template has a preset three-dimensional outline of the astronaut; performing image acquisition on the colored drawing template at a fixed position using an image acquisition device to obtain a target color image; and preprocessing the target color image to obtain a standard color image. By allowing users to doodle on the drawing template with colored pens, the present invention increases interactivity and participation, allowing users to directly participate in the design process. Users can freely select and combine colors, encouraging creative thinking, and enabling users to present their personalized ideas in a visual manner.

Description

Technical Field

[0001] The present invention relates to the field of virtual-reality interaction technology, and in particular to a virtual-reality interaction method and system based on image recognition technology. Background Art

[0002] Virtual-reality interaction refers to the combination of virtual environment and real environment to achieve real-time interaction between users and virtual objects. This interactive method uses technical means to integrate virtual elements such as digital information, images, and sounds with objects and environments in the real world to create an immersive experience.

[0003] Traditional methods usually rely on professional design tools, which makes it difficult for ordinary users to directly participate, limiting creative expression and personalized realization; traditional 3D modeling requires complex operations and professional knowledge, and the process is cumbersome and time-consuming, making it difficult to quickly generate the required model; and traditional methods may rely on manual color selection, which is prone to errors and causes the final effect to be inconsistent with user expectations; and traditional methods are often one-way, lacking real-time feedback and interaction, resulting in a relatively poor user experience. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a virtual-reality interaction method and system based on image recognition technology.

[0005] The technical solution adopted to solve the above technical problems is: a virtual-reality interaction method based on image recognition technology, including:

[0006] The target user uses a colored pen to color the drawing template to obtain a color drawing, and performs three-dimensional modeling of the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the drawing template is preset with the three-dimensional outline of the astronaut;

[0007] Capturing the color drawing at a fixed position using an image acquisition device to obtain a target color image, and preprocessing the target color image to obtain a standard color image;

[0008] Constructing a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model;

[0009] constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, performing image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0010] Based on the coordinate mapping table, coordinate mapping is performed on corresponding positions of the multiple different sub-standard color images in the astronaut three-dimensional model to obtain position mapping coordinates, color labels of corresponding positions in the astronaut three-dimensional model are obtained based on the position mapping coordinates, and colors corresponding to the color labels are rendered to the corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0011] Preferably, the number of the plurality of different sub-standard color images corresponds to the number of colors in the standard color image.

[0012] Preferably, preprocessing the target color image to obtain a standard color image includes:

[0013] The target color image is processed by a gamma unsharp masking algorithm to obtain a first color-enhanced image, and the first color-enhanced image is weighted to obtain a first weight. The calculation formula of the first weight is as follows:

[0014]

[0015] in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color-enhanced image, represents a variance of pixel values ​​of the first color-enhanced image;

[0016] The target color image is processed by a multi-scale Retinex and color restoration algorithm to obtain a second color-enhanced image, and a second weight of the second color-enhanced image is calculated. The calculation formula of the second weight is as follows:

[0017]

[0018] in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color-enhanced image, represents the variance of the pixel values ​​of the second color-enhanced image.

[0019] Preferably, preprocessing the target color image to obtain a standard color image further includes:

[0020] Based on the first weight, a pixel-level multiplication operation is performed on the first color-enhanced image through a Gaussian pyramid and a Laplacian pyramid to obtain a first output image. The expression of the first output image is as follows:

[0021]

[0022] in, Denotes the first output image, G l Represents the lth layer of the Gaussian pyramid, L l represents the lth level of the Laplace pyramid, represents the first weight after normalization;

[0023] A pixel-level multiplication operation is performed on the second color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the second weight to obtain a second output image. The expression of the second output image is as follows:

[0024]

[0025] in, represents the second output image, represents the normalized second weight;

[0026] The first output image and the second output image are reconstructed to obtain a standard color image, where the standard color image is expressed as follows:

[0027]

[0028] Among them, V(x,y) represents the standard color image.

[0029] Preferably, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system includes:

[0030] Obtain point cloud data of the astronaut three-dimensional model, and perform ground segmentation processing on the point cloud data to obtain ground points and segmentation points. The calculation formula for the ground segmentation processing is as follows:

[0031] Fangle=atan 2 (sinα, cosα);

[0032] Among them, Fangle represents flatness, atan represents the inverse tangent function, and α represents the angle between the center point and the adjacent points of the point cloud data;

[0033] Select any ground point and select multiple consecutive points in the vertical direction of the ground point to construct a point set. Calculate the smoothness of all point cloud data in the point set. When the smoothness is greater than a preset smoothness threshold, the point cloud data is a feature point. Otherwise, the point cloud data is a plane feature point. The smoothness calculation formula is as follows:

[0034]

[0035] Among them, c represents flatness, S represents point set, r i and r j Represents different continuous points selected in the vertical direction of the ground point.

[0036] Preferably, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system further includes:

[0037] Calculate the distance between the feature point and any of the plane feature points to obtain a distance set. Use the plane feature point corresponding to the minimum distance in the distance set as the matching point of the feature point. Traverse all feature points to obtain the matching point set corresponding to the point cloud data. The distance calculation formula is as follows:

[0038]

[0039] Where d represents the flatness, Indicates t k+1 The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, Indicates t k Coordinates of the feature points on the plane at that moment;

[0040] Obtaining a spatial coordinate set of the matching point set in the spatial coordinate system, and obtaining a plane coordinate set of the matching point set in the plane coordinate system;

[0041] A mapping function between the plane coordinate set and the space coordinate set is constructed, and a coordinate mapping table between the plane coordinate system and the space coordinate system is obtained based on the mapping function.

[0042] Preferably, the image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, the encoder block includes 1 overlapping block embedding and multiple hybrid Transformer blocks, and the number of hybrid Transformer blocks included in each encoder block from top to bottom is 3, 3, 3, 4, 6 and 3 respectively. A feature screening module is added to the encoder block in the lowest layer, and the feature screening module is used to screen low-dimensional features and retain features related to the target area. For the input standard color image, each encoder block extracts features of the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

[0043] Preferably, the decoder includes 6 decoder blocks, each of which includes a context Transformer block and a feature fusion module. The feature fusion module accepts feature maps output from the same dimension and all lower-dimensional encoder blocks, fuses all feature maps with feature maps output from the previous decoder block, processes them through the context Transformer block, and then upsamples them and outputs them to the next decoder block. The classifier determines the category of the standard color image to segment the standard color image.

[0044] Preferably, identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes:

[0045] Converting the plurality of different sub-standard color images from the RGB color space to the HSV color space;

[0046] Clustering the colors in the plurality of different sub-standard color images based on a K-means clustering algorithm to extract main colors;

[0047] Calculating a color histogram of the main color to count the frequency of each color in the multiple different sub-standard color images, and obtaining color values ​​of the multiple different sub-standard color images based on the frequency of each color in the multiple different sub-standard color images;

[0048] The color values ​​of the plurality of different sub-standard color images are compared with a preset color label library to obtain color labels of the plurality of different sub-standard color images.

[0049] The technical solution adopted to solve the above technical problems is: a virtual-reality interaction system based on image recognition technology, which is applicable to the virtual-reality interaction method based on image recognition technology, including:

[0050] a three-dimensional modeling unit, configured to perform three-dimensional modeling of the astronaut based on a three-dimensional modeling technique to obtain a three-dimensional model of the astronaut, wherein the drawing template is pre-set with a three-dimensional outline of the astronaut;

[0051] An image processing unit, configured to acquire an image of the color drawing at a fixed position based on an image acquisition device to obtain a target color image, and preprocess the target color image to obtain a standard color image;

[0052] a coordinate modeling unit configured to construct a coordinate system for the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system for the astronaut's three-dimensional model to obtain a space coordinate system corresponding to the astronaut's three-dimensional model;

[0053] a color extraction unit, the color extraction unit being configured to construct a coordinate mapping table between the plane coordinate system and the spatial coordinate system, perform image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0054] A virtual-reality interaction unit is configured to coordinately map the plurality of different sub-standard color images to corresponding positions in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtain color labels for corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on a color rendering technique.

[0055] The beneficial effects of the present invention are as follows: (1) The present invention increases interactivity and participation by allowing users to doodle on drawing templates with colored pens, allowing users to directly participate in the design process. Users can freely choose and combine colors, encouraging creative thinking, and allowing users' personalized ideas to be presented in a visual manner. Moreover, based on a preset three-dimensional model (such as an astronaut), a three-dimensional effect can be quickly generated, avoiding a complex modeling process and improving efficiency; (2) The present invention can ensure accurate color recognition and restoration by using image acquisition equipment and pre-processed standard color images, thereby improving the authenticity of the final effect, and by constructing a mapping relationship between a plane coordinate system and a space coordinate system, the user's doodle color can be accurately applied to the corresponding position of the three-dimensional model to achieve a realistic virtual effect; (3) The present invention improves the accuracy and efficiency of color recognition by applying a trained image segmentation model, so that multiple different sub-standard color images can be effectively processed and classified, and combined with color rendering technology, provides users with a more immersive experience, making the three-dimensional model more visually attractive and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic flow chart of the steps of the overall method in one embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the system architecture of the overall system in an embodiment of the present invention.

[0058] Figure numerals: 1. 3D modeling unit; 2. Image processing unit; 3. Coordinate modeling unit; 4. Color extraction unit; 5. Virtual-reality interaction unit. DETAILED DESCRIPTION

[0059] Example 1, as Figure 1 As shown, the present invention proposes a virtual-reality interaction method based on image recognition technology, comprising:

[0060] S1. The target user uses a colored pen to color a drawing template to obtain a color drawing, and then performs three-dimensional modeling of the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the drawing template has a preset three-dimensional outline of the astronaut;

[0061] S2. Capturing an image of a color drawing at a fixed position using an image acquisition device to obtain a target color image, and preprocessing the target color image to obtain a standard color image;

[0062] S3. Constructing a coordinate system for the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing a coordinate system for the astronaut's three-dimensional model to obtain a space coordinate system corresponding to the astronaut's three-dimensional model;

[0063] S4. Constructing a coordinate mapping table between a plane coordinate system and a spatial coordinate system, performing image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0064] S5. Based on the coordinate mapping table, coordinate mapping is performed on corresponding positions of the plurality of different sub-standard color images in the astronaut three-dimensional model to obtain position mapping coordinates, color labels of corresponding positions in the astronaut three-dimensional model are obtained based on the position mapping coordinates, and colors corresponding to the color labels are rendered to corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0065] In the present invention, three-dimensional modeling technology refers to the process of creating three-dimensional objects or scenes, using computer software to build, edit and optimize three-dimensional models; image acquisition equipment refers to equipment including cameras, scanners, etc. for capturing image information; a coordinate mapping table is a data structure used to map points in one coordinate system to another coordinate system for spatial conversion and position positioning; position mapping coordinates refer to the conversion of positions in a color drawing to corresponding positions in a three-dimensional model through a coordinate mapping table; color rendering refers to the process of applying color information to the surface of a three-dimensional model so that the model visually exhibits realistic color and material effects.

[0066] Embodiment 2, a virtual-reality interaction method based on image recognition technology proposed by the present invention, compared with embodiment 1, this embodiment further includes: the number of multiple different sub-standard color images corresponds to the number of colors in the standard color image.

[0067] In an optional embodiment, preprocessing the target color image to obtain a standard color image includes:

[0068] A1. Perform a gamma unsharp masking algorithm on the target color image to obtain a first color-enhanced image. Perform a weight calculation on the first color-enhanced image to obtain a first weight. The calculation formula of the first weight is as follows:

[0069]

[0070] in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color-enhanced image, represents a variance of pixel values ​​of the first color-enhanced image;

[0071] A2. Perform multi-scale Retinex and color restoration algorithm processing on the target color image to obtain a second color-enhanced image. Calculate a second weight of the second color-enhanced image. The calculation formula for the second weight is as follows:

[0072]

[0073] in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color-enhanced image, represents the variance of the pixel values ​​of the second color-enhanced image.

[0074] It should be noted that gamma unsharpening is an image enhancement technology that reduces the sharpening effect by adjusting the gamma value of the image, smoothes the details in the image, and enhances the visual effect; mask algorithm refers to the use of mask technology to select and process specific areas of the image, usually used to enhance or repair the image; multi-scale retinex refers to an image processing method that analyzes images at different scales to enhance brightness and color while reducing the impact of lighting changes; color restoration refers to the process of restoring the image color to a more realistic or expressive color, usually combined with other image enhancement techniques.

[0075] In an optional embodiment, preprocessing the target color image to obtain a standard color image further includes:

[0076] A3. Perform a pixel-level multiplication operation on the first color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the first weight to obtain a first output image. The expression of the first output image is as follows:

[0077]

[0078] in, Denotes the first output image, G l Represents the lth layer of the Gaussian pyramid, L l represents the lth level of the Laplace pyramid, represents the first weight after normalization;

[0079] A4. Perform a pixel-level multiplication operation on the second color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the second weight to obtain a second output image. The expression of the second output image is as follows:

[0080]

[0081] in, represents the second output image, represents the normalized second weight;

[0082] A5. Reconstruct the first output image and the second output image to obtain a standard color image. The expression of the standard color image is as follows:

[0083]

[0084] Among them, V(x,y) represents the standard color image.

[0085] It should be noted that the Gaussian pyramid refers to an image pyramid structure that generates a multi-scale image representation by blurring the image layer by layer; the Laplacian pyramid refers to extracting the edge and detail information of the image by differentiating each layer of the Gaussian pyramid, and is usually used for image reconstruction and enhancement; pixel-level multiplication refers to the operation of multiplying each pixel of the image one by one, and is usually used to combine two images or one image with weights.

[0086] In an optional embodiment, constructing a coordinate mapping table between a plane coordinate system and a space coordinate system includes:

[0087] B1. Obtain the point cloud data of the astronaut's 3D model and perform ground segmentation processing on the point cloud data to obtain ground points and segmentation points. The calculation formula for ground segmentation processing is as follows:

[0088] Fangle=atan 2 (sinα, cosα);

[0089] Among them, Fangle represents flatness, atan represents the inverse tangent function, and α represents the angle between the center point and the adjacent points of the point cloud data;

[0090] B2. Select any ground point and select multiple consecutive points in the vertical direction of the ground point to construct a point set. Calculate the smoothness of all point cloud data in the point set. If the smoothness is greater than the preset smoothness threshold, the point cloud data is a feature point. Otherwise, the point cloud data is a plane feature point. The smoothness calculation formula is as follows:

[0091]

[0092] Among them, c represents flatness, S represents point set, r i and r j Represents different continuous points selected in the vertical direction of the ground point.

[0093] In an optional embodiment, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system further includes:

[0094] B3. Calculate the distance between the feature point and any plane feature point to obtain a distance set. Use the plane feature point corresponding to the minimum distance in the distance set as the matching point of the feature point. Traverse all feature points to obtain the matching point set corresponding to the point cloud data. The distance calculation formula is as follows:

[0095]

[0096] Where d represents the flatness, Indicates t k+1 The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, Indicates t k Coordinates of the feature points on the plane at that moment;

[0097] B4. Obtain the spatial coordinate set of the matching point set in the spatial coordinate system, and obtain the plane coordinate set of the matching point set in the plane coordinate system;

[0098] B5. Construct a mapping function between the plane coordinate set and the space coordinate set, and obtain a coordinate mapping table between the plane coordinate system and the space coordinate system based on the mapping function.

[0099] In an optional embodiment, the image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, the encoder block includes 1 overlapping block embedding and multiple hybrid Transformer blocks, and the number of hybrid Transformer blocks included in each encoder block from top to bottom is 3, 3, 3, 4, 6 and 3 respectively. A feature screening module is added to the bottom-level encoder block, and the feature screening module is used to screen low-dimensional features and retain features related to the target area. For the input standard color image, each encoder block extracts features of the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

[0100] In an optional embodiment, the decoder includes 6 decoder blocks, the decoder block includes 1 context Transformer block and 1 feature fusion module, the feature fusion module accepts feature maps from the output of the same dimension and all lower dimensional encoder blocks, fuses all feature maps with the feature maps output by the previous layer decoder block, and upsamples them after processing by the context Transformer block, and outputs them to the next decoder block; the classifier judges the category of the standard color image to segment the standard color image.

[0101] It should be noted that the encoder extracts the features of the input image and converts high-dimensional data into low-dimensional representations, usually capturing important features through layer-by-layer processing; the decoder is responsible for gradually upsampling the low-dimensional feature map output by the encoder and restoring it to the resolution of the original image for subsequent processing; the classifier classifies the segmented areas and determines which category each area belongs to in order to achieve semantic segmentation of the image; the hybrid Transformer combines different types of attention mechanisms and convolutional network blocks to enhance feature extraction capabilities and utilize global and local information; downsampling reduces the spatial resolution of the feature map, usually through pooling or convolution operations to reduce computational complexity and extract high-level features; the context Transformer block uses the Transformer structure for processing contextual information to enhance feature relevance and context understanding capabilities; feature fusion combines feature maps from different encoder layers to retain multi-level information to better restore image details.

[0102] In an optional embodiment, identifying the colors of a plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes:

[0103] C1. Convert multiple different sub-standard color images from RGB color space to HSV color space;

[0104] C2. Cluster the colors in multiple different sub-standard color images based on the K-means clustering algorithm to extract the main colors;

[0105] C4. Calculating a color histogram of the main color to count the frequency of each color in the plurality of different sub-standard color images, and obtaining color values ​​of the plurality of different sub-standard color images based on the frequency of each color in the plurality of different sub-standard color images;

[0106] C5. Compare the color values ​​of the multiple different sub-standard color images with a preset color label library to obtain color labels of the multiple different sub-standard color images.

[0107] It should be noted that K-means clustering refers to an unsupervised learning algorithm designed to divide data points into K clusters. The algorithm iteratively minimizes the distance from each data point to the center of its cluster, thereby achieving data classification. In color clustering, it can be used to extract the main colors in an image. A color histogram is a tool that counts the frequency of each color in an image, usually represented as a bar graph. The height of each bar corresponding to a color represents the number of times or proportion that color appears in the image. Histograms can help analyze the color distribution of an image. A color label library is a predefined database containing colors and their corresponding labels (such as color names, categories, etc.). In image analysis, comparing extracted colors with colors in the label library can assign specific color labels to images for easy classification and identification.

[0108] Example 3, as Figure 2 As shown, the present invention proposes a virtual-reality interaction system based on image recognition technology, which is applicable to the virtual-reality interaction method based on image recognition technology, including:

[0109] The three-dimensional modeling unit 1 is used to perform three-dimensional modeling of the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template;

[0110] The image processing unit 2 is used to acquire an image of a color drawing at a fixed position based on an image acquisition device to obtain a target color image, and pre-process the target color image to obtain a standard color image;

[0111] A coordinate modeling unit 3 is used to construct a coordinate system for the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system for the astronaut's three-dimensional model to obtain a space coordinate system corresponding to the astronaut's three-dimensional model;

[0112] The color extraction unit 4 is used to construct a coordinate mapping table between a plane coordinate system and a space coordinate system, perform image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0113] The virtual-reality interaction unit 5 is used to coordinately map the corresponding positions of multiple different sub-standard color images in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtain color labels of corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0114] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A virtual-reality interaction method based on image recognition technology, characterized in that: include: The target user uses a colored pen to color the drawing template to obtain a color drawing, and performs three-dimensional modeling of the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the drawing template is preset with the three-dimensional outline of the astronaut; Capturing the color drawing at a fixed position using an image acquisition device to obtain a target color image, and preprocessing the target color image to obtain a standard color image; Constructing a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model; constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, performing image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images; performing coordinate mapping of corresponding positions of the plurality of different sub-standard color images in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtaining color labels for corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and rendering the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on a color rendering technique; Preprocessing the target color image to obtain a standard color image includes: Performing a gamma unsharp masking algorithm on the target color image to obtain a first color-enhanced image, and performing weight calculation on the first color-enhanced image to obtain a first weight; Performing multi-scale Retinex and color restoration algorithm processing on the target color image to obtain a second color-enhanced image, and calculating a second weight of the second color-enhanced image; performing a pixel-level multiplication operation on the first color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the first weight to obtain a first output image; performing a pixel-level multiplication operation on the second color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the second weight to obtain a second output image; The first output image and the second output image are reconstructed to obtain a standard color image.

2. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: The number of the plurality of different sub-standard color images corresponds to the number of colors in the standard color image.

3. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: The calculation formula of the first weight is as follows: ； in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color-enhanced image, represents a variance of pixel values ​​of the first color-enhanced image; The calculation formula of the second weight is as follows: ； in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color-enhanced image, represents the variance of the pixel values ​​of the second color-enhanced image.

4. The virtual-reality interaction method based on image recognition technology according to claim 3, characterized in that: The expression of the first output image is as follows: ； in, represents the first output image, represents the first layer, represents the first layer, represents the first weight after normalization; The expression of the second output image is as follows: ； in, represents the second output image, represents the normalized second weight; The expression of the standard color image is as follows: ； in, Represents a standard color image.

5. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, including: Obtain point cloud data of the astronaut three-dimensional model, and perform ground segmentation processing on the point cloud data to obtain ground points and segmentation points. The calculation formula for the ground segmentation processing is as follows: ； in, Indicates flatness, represents the inverse tangent function, Indicates the angle between the center point and adjacent points of the point cloud data; Select any ground point and select multiple consecutive points in the vertical direction of the ground point to construct a point set. Calculate the smoothness of all point cloud data in the point set. When the smoothness is greater than a preset smoothness threshold, the point cloud data is a feature point. Otherwise, the point cloud data is a plane feature point. The smoothness calculation formula is as follows: ； in, Indicates flatness, represents a point set, and Represents different continuous points selected in the vertical direction of the ground point.

6. The virtual-reality interaction method based on image recognition technology according to claim 5, characterized in that: Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system also includes: Calculate the distance between the feature point and any of the plane feature points to obtain a distance set. Use the plane feature point corresponding to the minimum distance in the distance set as the matching point of the feature point. Traverse all feature points to obtain the matching point set corresponding to the point cloud data. The distance calculation formula is as follows: ； in, Indicates flatness, express The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, express Coordinates of the feature points on the plane at that moment; Obtaining a spatial coordinate set of the matching point set in the spatial coordinate system, and obtaining a plane coordinate set of the matching point set in the plane coordinate system; A mapping function between the plane coordinate set and the space coordinate set is constructed, and a coordinate mapping table between the plane coordinate system and the space coordinate system is obtained based on the mapping function.

7. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: The image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, each of which includes 1 overlapping block embedding and multiple hybrid Transformer blocks. From top to bottom, the number of hybrid Transformer blocks included in each encoder block is 3, 3, 3, 4, 6 and 3 respectively. A feature screening module is added to the bottom-level encoder block. The feature screening module is used to screen low-dimensional features and retain features related to the target area. For the input standard color image, each encoder block extracts features of the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

8. The virtual-reality interaction method based on image recognition technology according to claim 7, characterized in that: The decoder includes 6 decoder blocks, each of which includes a context Transformer block and a feature fusion module. The feature fusion module accepts feature maps output from the same dimension and all lower-dimensional encoder blocks, fuses all feature maps with the feature maps output from the previous layer of decoder blocks, processes them through the context Transformer block, and then upsamples them and outputs them to the next decoder block. The classifier judges the category of the standard color image to segment the standard color image.

9. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: Identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes: Converting the plurality of different sub-standard color images from the RGB color space to the HSV color space; Clustering the colors in the plurality of different sub-standard color images based on a K-means clustering algorithm to extract main colors; Calculating a color histogram of the main color to count the frequency of each color in the multiple different sub-standard color images, and obtaining color values ​​of the multiple different sub-standard color images based on the frequency of each color in the multiple different sub-standard color images; The color values ​​of the plurality of different sub-standard color images are compared with a preset color label library to obtain color labels of the plurality of different sub-standard color images.

10. A virtual-reality interaction system based on image recognition technology, which is applicable to the virtual-reality interaction method based on image recognition technology according to any one of claims 1 to 9, characterized in that: include: A three-dimensional modeling unit (1), the three-dimensional modeling unit (1) is used to perform three-dimensional modeling on the astronaut based on a three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template; An image processing unit (2), the image processing unit (2) is used to perform image acquisition on the color drawing at a fixed position based on an image acquisition device to obtain a target color image, and pre-process the target color image to obtain a standard color image; A coordinate modeling unit (3), the coordinate modeling unit (3) is used to construct a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model; A color extraction unit (4), the color extraction unit (4) is used to construct a coordinate mapping table between the plane coordinate system and the space coordinate system, perform image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images; A virtual-real interaction unit (5), the virtual-real interaction unit (5) being used to coordinately map the plurality of different sub-standard color images to corresponding positions in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtain color labels of corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to corresponding positions in the astronaut three-dimensional model based on a color rendering technology; Preprocessing the target color image to obtain a standard color image includes: Performing a gamma unsharp masking algorithm on the target color image to obtain a first color-enhanced image, and performing weight calculation on the first color-enhanced image to obtain a first weight; Performing multi-scale Retinex and color restoration algorithm processing on the target color image to obtain a second color-enhanced image, and calculating a second weight of the second color-enhanced image; performing a pixel-level multiplication operation on the first color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the first weight to obtain a first output image; performing a pixel-level multiplication operation on the second color-enhanced image using a Gaussian pyramid and a Laplacian pyramid based on the second weight to obtain a second output image; The first output image and the second output image are reconstructed to obtain a standard color image.

Images