Virtual-real interaction method and system based on image recognition technology

Abstract

The invention relates to the technical field of virtual-real interaction, in particular to a virtual-real interaction method and system based on an image recognition technology, and the method comprises the steps: a target user carries out the color graffiti of a drawing template through a color pen, so as to obtain a color drawing, and carries out the three-dimensional modeling of a space man based on a three-dimensional modeling technology, so as to obtain a three-dimensional model of the space man, a three-dimensional outline of the space man is preset in the drawing template; and performing image acquisition on the color drawing at the fixed position based on an image acquisition device to obtain a target color image, and preprocessing the target color image to obtain a standard color image. According to the method, the user is allowed to use a color pen to scrawl on the drawing template, so that interactivity and sense of participation are increased, the user can directly participate in the design process, the user can freely select and combine colors and encourage creative thinking, and personalized ideas of the user can be presented in a visual mode.

Description

Technical Field

[0001] The present invention relates to the field of virtual-reality interaction technology, and in particular to a virtual-reality interaction method and system based on image recognition technology. Background Art

[0002] Virtual-reality interaction refers to the combination of the virtual environment and the real environment to achieve real-time interaction between users and virtual objects. This interactive method uses technical means to integrate virtual elements such as digital information, images, and sounds with objects and environments in the real world to create an immersive experience.

[0003] Traditional methods usually rely on professional design tools, which make it difficult for ordinary users to directly participate, limiting creative expression and personalized realization; traditional 3D modeling requires complex operations and expertise, and the process is cumbersome and time-consuming, making it difficult to quickly generate the required model; traditional methods may rely on manual color selection, which is prone to errors, resulting in the final effect not meeting user expectations; traditional methods are often one-way, lacking real-time feedback and interaction, and the user experience is relatively poor. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a virtual-reality interaction method and system based on image recognition technology.

[0005] The technical solution adopted to solve the above technical problems is: a virtual-reality interaction method based on image recognition technology, including:

[0006] The target user uses a colored pen to color the drawing template to obtain a color drawing, and performs three-dimensional modeling on the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template;

[0007] Based on the image acquisition device, the color drawing at a fixed position is imaged to obtain a target color image, and the target color image is preprocessed to obtain a standard color image;

[0008] Constructing the coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing the coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model;

[0009] Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, performing image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0010] Based on the coordinate mapping table, coordinate mapping is performed on corresponding positions of the multiple different sub-standard color images in the astronaut three-dimensional model to obtain position mapping coordinates, color labels of corresponding positions in the astronaut three-dimensional model are obtained based on the position mapping coordinates, and colors corresponding to the color labels are rendered to corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0011] Preferably, the number of the plurality of different sub-standard color images corresponds to the number of colors in the standard color image.

[0012] Preferably, preprocessing the target color image to obtain a standard color image includes:

[0013] The target color image is processed by a gamma unsharp mask algorithm to obtain a first color enhanced image, and the first color enhanced image is weighted to obtain a first weight, wherein the calculation formula of the first weight is as follows:

[0014]

[0015] in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color enhanced image, represents a variance of pixel values ​​of the first color enhanced image;

[0016] The target color image is processed by a multi-scale Retinex and color restoration algorithm to obtain a second color enhanced image, and a second weight of the second color enhanced image is calculated. The calculation formula of the second weight is as follows:

[0017]

[0018] in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color enhanced image, represents the variance of the pixel values ​​of the second color enhanced image.

[0019] Preferably, preprocessing the target color image to obtain a standard color image further includes:

[0020] Based on the first weight, the first color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a first output image. The expression of the first output image is as follows:

[0021]

[0022] in, represents the first output image, G l represents the lth layer of the Gaussian pyramid, L l represents the lth level of the Laplace pyramid, represents the first weight after normalization;

[0023] Based on the second weight, the second color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a second output image. The expression of the second output image is as follows:

[0024]

[0025] in, represents the second output image, represents the normalized second weight;

[0026] The first output image and the second output image are reconstructed to obtain a standard color image, and the expression of the standard color image is as follows:

[0027]

[0028] Among them, V(x,y) represents the standard color image.

[0029] Preferably, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system includes:

[0030] The point cloud data of the astronaut three-dimensional model is obtained, and the point cloud data is subjected to ground segmentation processing to obtain ground points and segmentation points. The calculation formula of the ground segmentation processing is as follows:

[0031] Fangle=atan 2 (sinα,cosα);

[0032] Among them, Fangle represents flatness, atan represents the inverse tangent function, and α represents the angle between the center point of the point cloud data and the adjacent points;

[0033] Select any one of the ground points, select multiple continuous points in the vertical direction of the ground point, construct a point set, calculate the smoothness of all point cloud data in the point set, when the smoothness is greater than the preset smoothness threshold, the point cloud data is a feature point, otherwise, the point cloud data is a plane feature point, the smoothness calculation formula is as follows:

[0034]

[0035] Among them, c represents flatness, S represents the point set, r i and r j Represents different continuous points selected in the vertical direction of the ground point.

[0036] Preferably, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system further includes:

[0037] The distance between the feature point and any one of the plane feature points is calculated to obtain a distance set, and the plane feature point corresponding to the smallest distance in the distance set is used as the matching point of the feature point. All feature points are traversed to obtain a matching point set corresponding to the point cloud data. The distance calculation formula is as follows:

[0038]

[0039] Where d represents the flatness, Indicates t k+1 The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, Indicates t k Coordinates of the feature points on the plane at the moment;

[0040] Acquire a spatial coordinate set of the matching point set in the spatial coordinate system, and acquire a plane coordinate set of the matching point set in the plane coordinate system;

[0041] A mapping function between the plane coordinate set and the space coordinate set is constructed, and a coordinate mapping table between the plane coordinate system and the space coordinate system is obtained based on the mapping function.

[0042] Preferably, the image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, the encoder block includes 1 overlapping block embedding and multiple hybrid Transformer blocks, the number of hybrid Transformer blocks included in each encoder block from top to bottom is 3, 3, 3, 4, 6 and 3 respectively, a feature screening module is added to the encoder block of the lowest layer, the feature screening module is used to screen low-dimensional features and retain features related to the target area, for the input standard color image, each encoder block performs feature extraction on the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

[0043] Preferably, the decoder includes 6 decoder blocks, each of which includes 1 context Transformer block and 1 feature fusion module, wherein the feature fusion module accepts feature maps output from encoder blocks of the same dimension and all lower dimensions, fuses all feature maps with feature maps output from the previous decoder block, processes them through the context Transformer block, and upsamples them before outputting them to the next decoder block; the classifier determines the category of the standard color image to segment the standard color image.

[0044] Preferably, identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes:

[0045] Converting the plurality of different sub-standard color images from the RGB color space to the HSV color space;

[0046] Clustering the colors in the plurality of different sub-standard color images based on a K-means clustering algorithm to extract main colors;

[0047] Calculating a color histogram of the main color to count the frequency of each color in the multiple different sub-standard color images, and obtaining color values ​​of the multiple different sub-standard color images based on the frequency of each color in the multiple different sub-standard color images;

[0048] The color values ​​of the plurality of different sub-standard color images are compared with a preset color label library to obtain color labels of the plurality of different sub-standard color images.

[0049] The technical solution adopted to solve the above technical problems is: a virtual-reality interaction system based on image recognition technology, which is applicable to the virtual-reality interaction method based on image recognition technology, including:

[0050] A three-dimensional modeling unit, the three-dimensional modeling unit is used to perform three-dimensional modeling on the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template;

[0051] An image processing unit, the image processing unit is used to perform image acquisition on the color drawing at a fixed position based on an image acquisition device to obtain a target color image, and pre-process the target color image to obtain a standard color image;

[0052] A coordinate modeling unit, the coordinate modeling unit is used to construct a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model;

[0053] A color extraction unit, the color extraction unit is used to construct a coordinate mapping table between the plane coordinate system and the space coordinate system, perform image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0054] A virtual-reality interaction unit, the virtual-reality interaction unit is used to perform coordinate mapping of the corresponding positions of the multiple different sub-standard color images in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtain color labels of the corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0055] The beneficial effects of the present invention are as follows: (1) The present invention increases interactivity and participation by allowing users to doodle on drawing templates with colored pens, allowing users to directly participate in the design process, and users can freely choose and combine colors, encouraging creative thinking, and allowing users' personalized ideas to be presented in a visual manner. Moreover, based on a preset three-dimensional model (such as an astronaut), a three-dimensional effect can be quickly generated, avoiding a complex modeling process and improving efficiency; (2) The present invention can ensure accurate color recognition and restoration by using an image acquisition device and a pre-processed standard color image, thereby improving the authenticity of the final effect, and by constructing a mapping relationship between a plane coordinate system and a space coordinate system, the user's doodle color can be accurately applied to the corresponding position of the three-dimensional model to achieve a real virtual effect; (3) The present invention improves the accuracy and efficiency of color recognition by applying a trained image segmentation model, so that multiple different sub-standard color images can be effectively processed and classified, and combined with color rendering technology, a more immersive experience is provided to users, making the three-dimensional model more visually attractive and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic flow chart of the steps of an overall method in an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of the system architecture of an overall system in an embodiment of the present invention.

[0058] Figure numerals: 1. 3D modeling unit; 2. Image processing unit; 3. Coordinate modeling unit; 4. Color extraction unit; 5. Virtual-reality interaction unit. DETAILED DESCRIPTION

[0059] Embodiment 1, as Figure 1 As shown, the present invention proposes a virtual-reality interaction method based on image recognition technology, comprising:

[0060] S1. The target user uses a colored pen to color the drawing template to obtain a color drawing, and performs three-dimensional modeling of the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the drawing template has a preset three-dimensional outline of the astronaut;

[0061] S2, performing image acquisition on a color drawing at a fixed position based on an image acquisition device to obtain a target color image, and preprocessing the target color image to obtain a standard color image;

[0062] S3. constructing a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model;

[0063] S4, constructing a coordinate mapping table between a plane coordinate system and a space coordinate system, performing image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0064] S5. Based on the coordinate mapping table, coordinate mapping is performed on corresponding positions of multiple different sub-standard color images in the astronaut three-dimensional model to obtain position mapping coordinates, color labels of corresponding positions in the astronaut three-dimensional model are obtained based on the position mapping coordinates, and colors corresponding to the color labels are rendered to corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0065] In the present invention, 3D modeling technology refers to the process of creating 3D objects or scenes, using computer software to build, edit and optimize 3D models; image acquisition equipment refers to equipment including cameras, scanners, etc., for capturing image information; a coordinate mapping table is a data structure used to map points in one coordinate system to another coordinate system for spatial conversion and position positioning; position mapping coordinates refer to the conversion of positions in a color drawing to corresponding positions in a 3D model through a coordinate mapping table; color rendering refers to the process of applying color information to the surface of a 3D model so that the model visually exhibits real color and material effects.

[0066] Embodiment 2, a virtual-reality interaction method based on image recognition technology proposed by the present invention, compared with embodiment 1, this embodiment also includes: the number of multiple different sub-standard color images corresponds to the number of colors in the standard color image.

[0067] In an optional embodiment, preprocessing the target color image to obtain a standard color image includes:

[0068] A1. Process the target color image using a gamma unsharp mask algorithm to obtain a first color enhanced image. Perform weight calculation on the first color enhanced image to obtain a first weight. The calculation formula of the first weight is as follows:

[0069]

[0070] in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color enhanced image, represents a variance of pixel values ​​of the first color enhanced image;

[0071] A2. Perform multi-scale Retinex and color restoration algorithm processing on the target color image to obtain a second color enhanced image, and calculate the second weight of the second color enhanced image. The calculation formula of the second weight is as follows:

[0072]

[0073] in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color enhanced image, represents the variance of the pixel values ​​of the second color enhanced image.

[0074] It should be noted that gamma unsharpening is an image enhancement technology that reduces the sharpening effect, smoothes the details in the image and enhances the visual effect by adjusting the gamma value of the image; mask algorithm refers to the use of mask technology to select and process specific areas of the image, usually used to enhance or repair the image; multi-scale Retinex refers to an image processing method that analyzes images at different scales to enhance brightness and color while reducing the impact of lighting changes; color restoration refers to the process of restoring the color of an image to a more realistic or expressive state, usually in combination with other image enhancement techniques.

[0075] In an optional embodiment, preprocessing the target color image to obtain a standard color image further includes:

[0076] A3. Based on the first weight, the first color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a first output image. The expression of the first output image is as follows:

[0077]

[0078] in, represents the first output image, G l represents the lth layer of the Gaussian pyramid, L l represents the lth level of the Laplace pyramid, represents the first weight after normalization;

[0079] A4. Based on the second weight, the second color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a second output image. The expression of the second output image is as follows:

[0080]

[0081] in, represents the second output image, represents the normalized second weight;

[0082] A5. Reconstruct the first output image and the second output image to obtain a standard color image. The expression of the standard color image is as follows:

[0083]

[0084] Among them, V(x,y) represents the standard color image.

[0085] It should be noted that Gaussian pyramid refers to an image pyramid structure, which generates a multi-scale image representation by blurring the image layer by layer; Laplacian pyramid refers to extracting the edge and detail information of the image by differentiating each layer of the Gaussian pyramid, and is usually used for image reconstruction and enhancement; pixel-level multiplication refers to the operation of multiplying each pixel of the image one by one, and is usually used to combine two images or an image with weights.

[0086] In an optional embodiment, constructing a coordinate mapping table between a plane coordinate system and a space coordinate system includes:

[0087] B1. Obtain the point cloud data of the astronaut's three-dimensional model and perform ground segmentation processing on the point cloud data to obtain ground points and segmentation points. The calculation formula for ground segmentation processing is as follows:

[0088] Fangle=atan 2 (sinα,cosα);

[0089] Among them, Fangle represents flatness, atan represents the inverse tangent function, and α represents the angle between the center point of the point cloud data and the adjacent points;

[0090] B2. Select any ground point, select multiple continuous points in the vertical direction of the ground point, construct a point set, and calculate the smoothness of all point cloud data in the point set. When the smoothness is greater than the preset smoothness threshold, the point cloud data is a feature point, otherwise, the point cloud data is a plane feature point. The smoothness calculation formula is as follows:

[0091]

[0092] Among them, c represents flatness, S represents the point set, r i and r j Represents different continuous points selected in the vertical direction of the ground point.

[0093] In an optional embodiment, constructing a coordinate mapping table between the plane coordinate system and the space coordinate system further includes:

[0094] B3. Calculate the distance between the feature point and any plane feature point to obtain a distance set. Take the plane feature point corresponding to the smallest distance in the distance set as the matching point of the feature point. Traverse all feature points to obtain the matching point set corresponding to the point cloud data. The distance calculation formula is as follows:

[0095]

[0096] Where d represents the flatness, Indicates t k+1 The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, Indicates t k Coordinates of the feature points on the plane at the moment;

[0097] B4. Obtain a spatial coordinate set of the matching point set in the spatial coordinate system, and obtain a plane coordinate set of the matching point set in the plane coordinate system;

[0098] B5. Construct a mapping function between the plane coordinate set and the space coordinate set, and obtain a coordinate mapping table between the plane coordinate system and the space coordinate system based on the mapping function.

[0099] In an optional embodiment, the image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, the encoder block includes 1 overlapping block embedding and multiple hybrid Transformer blocks, the number of hybrid Transformer blocks included in each encoder block from top to bottom is 3, 3, 3, 4, 6 and 3 respectively, a feature screening module is added to the bottom-level encoder block, the feature screening module is used to screen low-dimensional features and retain features related to the target area, for the input standard color image, each encoder block extracts features of the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

[0100] In an optional embodiment, the decoder includes 6 decoder blocks, the decoder block includes 1 context Transformer block and 1 feature fusion module, the feature fusion module accepts feature maps from the same dimension and all lower dimensional encoder blocks output, fuses all feature maps with feature maps output by the previous decoder block, and upsamples them after processing by the context Transformer block, and outputs them to the next decoder block; the classifier judges the category of the standard color image to segment the standard color image.

[0101] It should be noted that the encoder extracts the features of the input image and converts high-dimensional data into low-dimensional representations, usually capturing important features through layer-by-layer processing; the decoder is responsible for gradually upsampling the low-dimensional feature map output by the encoder and restoring it to the resolution of the original image for subsequent processing; the classifier classifies the segmented areas and determines which category each area belongs to in order to achieve semantic segmentation of the image; the hybrid Transformer combines different types of attention mechanisms and convolutional network blocks to enhance feature extraction capabilities and utilize global and local information; downsampling reduces the spatial resolution of feature maps, usually through pooling or convolution operations to reduce computational complexity and extract high-level features; the contextual Transformer block uses the Transformer structure for processing using contextual information to enhance feature relevance and context understanding capabilities; feature fusion combines feature maps from different encoder layers to retain multi-level information in order to better restore image details.

[0102] In an optional embodiment, identifying the colors of a plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes:

[0103] C1. Convert multiple different sub-standard color images from RGB color space to HSV color space;

[0104] C2, clustering the colors in multiple different sub-standard color images based on the K-means clustering algorithm to extract the main colors;

[0105] C4. Calculating a color histogram of the main color to count the frequency of each color in a plurality of different sub-standard color images, and obtaining color values ​​of the plurality of different sub-standard color images based on the frequency of each color in the plurality of different sub-standard color images;

[0106] C5. Compare the color values ​​of the multiple different sub-standard color images with a preset color label library to obtain color labels of the multiple different sub-standard color images.

[0107] It should be noted that K-means clustering refers to an unsupervised learning algorithm that aims to divide data points into K clusters. The algorithm minimizes the distance from each data point to the center of its cluster in an iterative manner, thereby achieving data classification. In color clustering, it can be used to extract the main colors in an image; the color histogram is a tool that counts the frequency of each color in an image, usually in the form of a bar graph. The height of the bar corresponding to each color represents the number of times or proportion of the color in the image. The histogram can help analyze the color distribution of the image; the color label library is a predefined database containing colors and their corresponding labels (such as color names, categories, etc.). In image analysis, the extracted colors are compared with the colors in the label library to give specific color labels to the image for easy classification and identification.

[0108] Embodiment three, as Figure 2 As shown, the present invention proposes a virtual-reality interaction system based on image recognition technology, which is applicable to the virtual-reality interaction method based on image recognition technology, including:

[0109] A three-dimensional modeling unit 1 is used to perform three-dimensional modeling on the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template;

[0110] The image processing unit 2 is used to acquire an image of a color drawing at a fixed position based on an image acquisition device to obtain a target color image, and to pre-process the target color image to obtain a standard color image;

[0111] A coordinate modeling unit 3, the coordinate modeling unit 3 is used to construct a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model;

[0112] A color extraction unit 4 is used to construct a coordinate mapping table between a plane coordinate system and a space coordinate system, perform image segmentation on the standard color image based on a trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images;

[0113] The virtual-reality interaction unit 5 is used to coordinately map the corresponding positions of a plurality of different sub-standard color images in the astronaut three-dimensional model based on a coordinate mapping table to obtain position mapping coordinates, obtain color labels of corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on color rendering technology.

[0114] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A virtual-reality interaction method based on image recognition technology, characterized in that: include: The target user uses a colored pen to color the drawing template to obtain a color drawing, and performs three-dimensional modeling on the astronaut based on the three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template; Based on the image acquisition device, the color drawing at a fixed position is imaged to obtain a target color image, and the target color image is preprocessed to obtain a standard color image; Constructing the coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and constructing the coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model; Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, performing image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images; Based on the coordinate mapping table, coordinate mapping is performed on corresponding positions of the multiple different sub-standard color images in the astronaut three-dimensional model to obtain position mapping coordinates, color labels of corresponding positions in the astronaut three-dimensional model are obtained based on the position mapping coordinates, and colors corresponding to the color labels are rendered to corresponding positions in the astronaut three-dimensional model based on color rendering technology.

2. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: The number of the plurality of different sub-standard color images corresponds to the number of colors in the standard color image.

3. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: Preprocessing the target color image to obtain a standard color image includes: The target color image is processed by a gamma unsharp mask algorithm to obtain a first color enhanced image, and the first color enhanced image is weighted to obtain a first weight, wherein the calculation formula of the first weight is as follows: in, represents the first weight, represents the index of the first color-enhanced image, represents the average value of the pixel values ​​of the first color enhanced image, represents a variance of pixel values ​​of the first color enhanced image; The target color image is processed by a multi-scale Retinex and color restoration algorithm to obtain a second color enhanced image, and a second weight of the second color enhanced image is calculated. The calculation formula of the second weight is as follows: in, represents the second weight, represents the index of the second color-enhanced image, represents the average value of the pixel values ​​of the second color enhanced image, represents the variance of the pixel values ​​of the second color enhanced image.

4. The virtual-reality interaction method based on image recognition technology according to claim 3 is characterized in that: Preprocessing the target color image to obtain a standard color image also includes: Based on the first weight, the first color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a first output image. The expression of the first output image is as follows: in, represents the first output image, G l represents the lth layer of the Gaussian pyramid, L l represents the lth level of the Laplace pyramid, represents the first weight after normalization; Based on the second weight, the second color enhanced image is multiplied pixel by pixel through a Gaussian pyramid and a Laplacian pyramid to obtain a second output image. The expression of the second output image is as follows: in, represents the second output image, represents the normalized second weight; The first output image and the second output image are reconstructed to obtain a standard color image, and the expression of the standard color image is as follows: Among them, V(x,y) represents the standard color image.

5. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system, comprising: The point cloud data of the astronaut three-dimensional model is obtained, and the point cloud data is subjected to ground segmentation processing to obtain ground points and segmentation points. The calculation formula of the ground segmentation processing is as follows: Fangle=atan 2 (sinα,cosα); Among them, Fangle represents flatness, atan represents the inverse tangent function, and α represents the angle between the center point of the point cloud data and the adjacent points; Select any one of the ground points, select multiple continuous points in the vertical direction of the ground point, construct a point set, calculate the smoothness of all point cloud data in the point set, when the smoothness is greater than the preset smoothness threshold, the point cloud data is a feature point, otherwise, the point cloud data is a plane feature point, the smoothness calculation formula is as follows: Among them, c represents flatness, S represents the point set, r i and r j Represents different continuous points selected in the vertical direction of the ground point.

6. The virtual-reality interaction method based on image recognition technology according to claim 5, characterized in that: Constructing a coordinate mapping table between the plane coordinate system and the space coordinate system also includes: The distance between the feature point and any one of the plane feature points is calculated to obtain a distance set, and the plane feature point corresponding to the smallest distance in the distance set is used as the matching point of the feature point. All feature points are traversed to obtain a matching point set corresponding to the point cloud data. The distance calculation formula is as follows: Where d represents the flatness, Indicates t k+1 The coordinates of the feature points at the moment, express The coordinates of the nearest feature point, Indicates t k Coordinates of the feature points on the plane at the moment; Acquire a spatial coordinate set of the matching point set in the spatial coordinate system, and acquire a plane coordinate set of the matching point set in the plane coordinate system; A mapping function between the plane coordinate set and the space coordinate set is constructed, and a coordinate mapping table between the plane coordinate system and the space coordinate system is obtained based on the mapping function.

7. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: The image segmentation model includes an encoder, a decoder and a classifier; the encoder includes 6 encoder blocks, each of which includes 1 overlapping block embedding and multiple hybrid Transformer blocks. The number of hybrid Transformer blocks included in each encoder block from top to bottom is 3, 3, 3, 4, 6 and 3 respectively. A feature screening module is added to the encoder block of the lowest layer. The feature screening module is used to screen low-dimensional features and retain features related to the target area. For the input standard color image, each encoder block extracts features of the standard color image, downsamples the processed feature map and outputs it to the next encoder block.

8. The virtual-reality interaction method based on image recognition technology according to claim 7, characterized in that: The decoder includes 6 decoder blocks, each of which includes 1 context Transformer block and 1 feature fusion module. The feature fusion module accepts feature maps output from encoder blocks of the same dimension and all lower dimensions, fuses all feature maps with feature maps output from the decoder block of the previous layer, and upsamples them after being processed by the context Transformer block, and outputs them to the next decoder block. The classifier determines the category of the standard color image to segment the standard color image.

9. The virtual-reality interaction method based on image recognition technology according to claim 1, characterized in that: Identifying the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images includes: Converting the plurality of different sub-standard color images from the RGB color space to the HSV color space; Clustering the colors in the plurality of different sub-standard color images based on a K-means clustering algorithm to extract main colors; Calculating a color histogram of the main color to count the frequency of each color in the multiple different sub-standard color images, and obtaining color values ​​of the multiple different sub-standard color images based on the frequency of each color in the multiple different sub-standard color images; The color values ​​of the plurality of different sub-standard color images are compared with a preset color label library to obtain color labels of the plurality of different sub-standard color images.

10. A virtual-reality interaction system based on image recognition technology, which is applicable to a virtual-reality interaction method based on image recognition technology according to any one of claims 1 to 9, characterized in that: include: A three-dimensional modeling unit (1), the three-dimensional modeling unit (1) being used to perform three-dimensional modeling on an astronaut based on a three-dimensional modeling technology to obtain a three-dimensional model of the astronaut, wherein the three-dimensional outline of the astronaut is preset in the drawing template; An image processing unit (2), the image processing unit (2) being used for performing image acquisition on the color drawing at a fixed position based on an image acquisition device to obtain a target color image, and performing preprocessing on the target color image to obtain a standard color image; A coordinate modeling unit (3), the coordinate modeling unit (3) being used to construct a coordinate system of the standard color image to obtain a plane coordinate system corresponding to the standard color image, and to construct a coordinate system of the astronaut three-dimensional model to obtain a space coordinate system corresponding to the astronaut three-dimensional model; A color extraction unit (4), the color extraction unit (4) is used to construct a coordinate mapping table between the plane coordinate system and the space coordinate system, perform image segmentation on the standard color image based on the trained image segmentation model to obtain a plurality of different sub-standard color images, and identify the colors of the plurality of different sub-standard color images to obtain color labels corresponding to the plurality of different sub-standard color images; A virtual-real interaction unit (5), the virtual-real interaction unit (5) being used to coordinately map the corresponding positions of the plurality of different sub-standard color images in the astronaut three-dimensional model based on the coordinate mapping table to obtain position mapping coordinates, obtain color labels of the corresponding positions in the astronaut three-dimensional model based on the position mapping coordinates, and render the colors corresponding to the color labels to the corresponding positions in the astronaut three-dimensional model based on color rendering technology.

Images