Map fusion method of VR device and electronic device

Abstract

The application provides a map fusion method of a VR device and an electronic device, comprising: extracting feature point sets of target images respectively photographed by any two VR devices; determining whether the same target objects exist in the two target images based on the result of feature matching of the two feature point sets; if so, extracting color information of each feature point in the corresponding feature point set from the two target images, determining color similarity of the two feature points for color information of any two feature points with the same position coordinates in the same world coordinate system in the two feature point sets; if the color similarity is greater than a first specified threshold, determining that the two feature points are matched; if any two feature points with the same position coordinates in the two feature point sets are matched, determining that the poses of the two VR devices are the same, and fusing the respective maps of the two VR devices based on the poses of the two VR devices. The accuracy of the map fusion of the VR device is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality device positioning, and particularly relates to a map fusion method of a VR device and an electronic device. BACKGROUND

[0002] Map fusion of two virtual reality (VR) devices is achieved based on visual simultaneous localization and mapping (SLAM) technology. In the visual SLAM technology, image matching is a key basic technology. The most commonly used image matching is achieved based on image feature point matching technology, and therefore the accuracy of image feature point matching directly affects the positioning accuracy of the visual SLAM system and further affects the accuracy of map fusion of the two VR devices.

[0003] In the prior art, image feature points are matched based on the gray histogram of each image to extract feature points, and then the feature points of two images are matched through position coordinates. However, in the case of different colors but the same scene, the feature points extracted for each scene and the position coordinates of the extracted feature points are the same. For example, in two rooms, there is furniture of the same style, but the colors of the furniture are different. Using the prior art method to extract feature points, the result is that the feature points extracted in the two rooms are the same, and the positions of the feature points are also the same. Therefore, the feature points in different scenes are matched, which causes the prior art method to have the problem of feature point mismatching, and reduces the accuracy of map fusion of the VR device. SUMMARY

[0004] The present application provides a map fusion method of a VR device and an electronic device, which is used to solve the problem of feature point mismatching in the prior art and improve the accuracy of map fusion of the VR device.

[0005] In a first aspect, an embodiment of the present application provides a map fusion method of a VR device, comprising:

[0006] obtaining target images respectively photographed by any two VR devices, and extracting feature point sets in the target images;

[0007] performing feature matching on the two feature point sets, and determining whether there is a target object of the same form in the two target images based on a matching result;

[0008] if the two target images exist, color information of each feature point in the corresponding feature point set is extracted from the two target images respectively, and color similarity between any two feature points with the same position coordinates in the two feature point sets is determined according to color information of the two feature points; wherein the position coordinates are position coordinates in a same world coordinate system; and

[0009] if the color similarity between the two feature points is greater than a first specified threshold, it is determined that the two feature points are matched;

[0010] if any two feature points with the same position coordinates in the two feature point sets are matched, it is determined that poses of the two VR devices are the same, and maps corresponding to the two VR devices respectively are fused based on the poses of the two VR devices.

[0011] The second aspect of the application provides an electronic device, comprising a processor and a memory, the processor and the memory are connected through a bus;

[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0013] target images photographed by any two VR devices are acquired, and feature point sets in the target images are extracted respectively;

[0014] feature matching is performed on the two feature point sets, and whether there is a target object with the same shape in the two target images is determined based on a matching result;

[0015] if the two target images exist, color information of each feature point in the corresponding feature point set is extracted from the two target images respectively, and color similarity between any two feature points with the same position coordinates in the two feature point sets is determined according to color information of the two feature points; wherein the position coordinates are position coordinates in a same world coordinate system; and

[0016] if the color similarity between the two feature points is greater than a first specified threshold, it is determined that the two feature points are matched;

[0017] if any two feature points with the same position coordinates in the two feature point sets are matched, it is determined that poses of the two VR devices are the same, and maps corresponding to the two VR devices respectively are fused based on the poses of the two VR devices.

[0018] According to the third aspect provided by the embodiment of the application, a computer storage medium is provided, the computer storage medium stores a computer program, and the computer program is used to execute the method of the first aspect.

[0019] In the above embodiments of the present application, under the premise that it is determined that the same object in the same form exists in the target images respectively photographed by the two VR devices, the color information of each feature point in the respective feature point sets extracted from the two target images is extracted, and the color similarity between any two feature points with the same position coordinates in the two feature point sets is determined, and then whether the two feature points match is determined through the color similarity of the two feature points, so as to realize the map fusion of the two VR devices. Thus, in the embodiment, after it is determined that the same target object in the same form exists in the two target images, the feature point matching needs to be performed in combination with the color information of each feature point in the target images. Thus, the problem of false matching in the case of the same scene but different colors is avoided, and the accuracy of the map fusion of the two VR devices is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 An application scenario provided by the embodiments of the present application is exemplarily shown;

[0022] Figure 2 A flowchart of a map fusion method of a VR device provided by the embodiments of the present application is exemplarily shown;

[0023] Figure 3 A feature point diagram of a target image provided by the embodiments of the present application is exemplarily shown;

[0024] Figure 4 A flowchart of determining the color similarity of a feature point provided by the embodiments of the present application is exemplarily shown;

[0025] Figure 5 A flowchart of map fusion provided by the embodiments of the present application is exemplarily shown;

[0026] Figure 6 A flowchart of a map fusion method of a VR device provided by the embodiments of the present application is exemplarily shown;

[0027] Figure 7 A structure diagram of a map fusion device of a VR device provided by the embodiments of the present application is exemplarily shown;

[0028] Figure 8 A hardware structure diagram of a calibration device provided by the embodiments of the present application is exemplarily shown. DETAILED DESCRIPTION

[0029] For the purpose of making the objects, implementations and advantages of the present application more clear, the following will make a clear and complete description of the exemplary embodiments of the present application in combination with the drawings of the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, but not all the embodiments.

[0030] Based on the exemplary embodiments described in the present application, all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of the claims of the present application. In addition, although the disclosure in the present application is introduced according to one or several examples, it should be understood that each aspect of the disclosure can also constitute a complete embodiment independently.

[0031] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, but is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0032] The terms "first", "second", and the like in the specification of the present application, the claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not necessarily limit to those components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0033] The term "module" used in the present application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code capable of performing functions related to the element.

[0034] The idea of the embodiments of the present application is summarized as follows.

[0035] In the current feature point matching technology, the matching of image feature points is based on the extraction of feature points from the gray histogram of each image, and then the feature points of two images are matched through position coordinates. However, for the case of different colors but the same scene, the feature points extracted for each scene and the position coordinates of the extracted feature points are the same. For example, there are furniture of the same style in two rooms, but the colors of the furniture are different. Using the existing technology to extract feature points, the result is that the feature points extracted in the two rooms are the same, and the positions of the feature points are also the same. Therefore, the feature points in different scenes may be matched. The existing technology has the problem of feature point mismatch, which reduces the accuracy of map fusion of the VR device.

[0036] Based on the problem of mismatch in the prior art, the embodiment of the present application provides a map fusion method of a VR device. Under the premise that it is determined that there are objects of the same shape in the target images photographed by two VR devices respectively, color information of each feature point in the corresponding feature point set is extracted from the two target images respectively, and for the color information of any two feature points with the same position coordinates in the two feature point sets, the color similarity between the two feature points is determined, and then whether the two feature points are matched is determined through the color similarity of the two feature points, so as to realize the map fusion of the two VR devices. Therefore, in the embodiment, after it is determined that there are target objects of the same shape in the two target images, the feature point matching needs to be combined with the color information of each feature point in the target image. Therefore, the problem of mismatch in the case of the same scene but different colors is avoided, and the accuracy of map fusion of the two VR devices is improved.

[0037] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0038] Figure 1 An application scenario provided by the embodiment of the present application is exemplarily shown. As shown in Figure 1 The application scenario is illustrated by taking an electronic device as a server as an example. The application scenario includes a VR device 110, a VR device 120 and a server 130. The server 130 can be implemented by a single server or multiple servers. The server 130 can be implemented by a physical server or a virtual server.

[0039] In one possible application scenario, VR devices 110 and 120 respectively send the captured target images to server 130. After obtaining the target images captured by VR devices 110 and 120, server 130 extracts feature point sets from the target images respectively, performs feature matching on the two feature point sets, and then determines whether there are target objects with the same shape in the two target images based on the matching results. If server 130 determines that there are target objects with the same shape in the two target images, it extracts the color information of each feature point in the corresponding feature point set from the two target images respectively, and determines the color similarity between the two feature points for any two feature points with the same position coordinates in the two feature point sets; and if the color similarity between the two feature points is greater than a first specified threshold, it determines that the two feature points are matched. Finally, if the server 130 determines that any two feature points with the same position coordinates in the two feature point sets are matched, it determines that the poses of the two VR devices are the same, and merges the maps corresponding to the two VR devices based on the poses of the two VR devices, and sends the merged map to VR device 110 and VR device 120 so that VR device 110 and VR device 120 can update their respective maps using the merged map.

[0040] It should be noted that the position coordinates of each feature point in this embodiment are all position coordinates in the same world coordinate system. Furthermore, the style of the VR device in this embodiment is for illustrative purposes only and does not limit the actual style of the VR device.

[0041] like Figure 2 The diagram shown illustrates a map fusion method for VR devices, which may include the following steps:

[0042] Step 201: Obtain target images captured by any two VR devices, and extract feature point sets from the target images respectively;

[0043] like Figure 3 The image shown is a schematic diagram after feature points have been extracted from the target image. Figure 3 As can be seen, the black dots in the image extract a large number of feature points from the target image sofa, and all the feature points extracted from the target image constitute the feature point set of the target image.

[0044] In this embodiment, the feature point extraction algorithm can be the ORB (Oriented Fast and Rotated BRIEF) feature extraction algorithm. The ORB feature point extraction algorithm in this embodiment is only for illustrative purposes. The specific feature extraction algorithm can be selected according to the actual situation, and this embodiment does not limit the feature point extraction algorithm.

[0045] Step 202: performing feature matching on the two feature point sets, and determining whether there is a target object with the same shape in the two target images based on the matching result;

[0046] In an embodiment, the matching result is determined by the following way:

[0047] For any feature point in any one of the two feature point sets, if there is a feature point with the same position coordinates in the other feature point set, it is determined that the matching result is that the two feature point sets match; otherwise, it is determined that the matching result is that the two feature point sets do not match.

[0048] For example, feature point set 1 includes feature point A, feature point B, feature point C and feature point D. Feature point set 2 includes feature point M, feature point N, feature point O and feature point P. If the position coordinates of feature point A are the same as the position coordinates of feature point P, the position coordinates of feature point B are the same as the position coordinates of feature point O, the position coordinates of feature point C are the same as the position coordinates of feature point M, and the position coordinates of feature point D are the same as the position coordinates of feature point N, it is determined that the matching result is that feature point set 1 and feature point set 2 match.

[0049] It should be noted that the position coordinates of each feature point in this embodiment and the context are all position coordinates in the world coordinate system. Moreover, the number of feature points in the feature point set in this embodiment is subject to the actual extracted feature points, and this embodiment does not limit the number of feature points in the feature point set.

[0050] The determination of whether there is a target object with the same shape in the two target images based on the matching result in step 202 can be specifically implemented as follows: if the matching result is that the two feature point sets match, it is determined that there is a target object with the same shape in the two target images; if the matching result is that the two feature point sets do not match, it is determined that there is no target object with the same shape in the two target images.

[0051] Step 203: if there is, color information of each feature point in the corresponding feature point set is extracted from the two target images respectively, and color similarity between any two feature points with the same position coordinates in the two feature point sets is determined based on the color information of the two feature points; wherein the position coordinates are position coordinates in the same world coordinate system.

[0052] In this embodiment, the color information is an RGB color value, and the RGB color value is composed of a parameter value corresponding to a red channel, a parameter value corresponding to a green channel and a parameter value corresponding to a blue channel. Specifically, the RGB color value of a feature point can be obtained by the following way:

[0053] The image coordinate values of each feature point in the target image are extracted by using a feature point extraction algorithm, and then the RGB color values of each feature point are determined from the target image based on the image coordinate values of each feature point.

[0054] Since the feature point extraction algorithm needs to convert the target image into a grayscale histogram, the RGB color values of the feature points cannot be determined by using the feature point extraction algorithm, and the RGB color values of the feature points are determined from the original target image based on the image coordinate values of each feature point.

[0055] As shown in FIG. 3, it is a flowchart for determining the color similarity between two feature points in step 203, which can include the following steps: Figure 4

[0056] Step 2031: converting the RGB color values of the two feature points into corresponding HSV color values, respectively;

[0057] Specifically, for any one feature point, the following steps are performed:

[0058] determining the maximum parameter value in the RGB color value as the lightness parameter value in the HSV color value, and obtaining the saturation parameter value in the HSV color value according to the maximum parameter value and the minimum parameter value in the RGB color value, and obtaining the hue parameter value in the HSV color value through the respective parameter values corresponding to the RGB color value, and obtaining the HSV color value based on the lightness parameter value, the saturation parameter value and the hue parameter value.

[0059] Next, the ways of determining the lightness parameter value, the saturation parameter value and the hue parameter value are introduced respectively:

[0060] 1. Saturation parameter value S

[0061] subtracting the minimum parameter value from the maximum parameter value to obtain a parameter difference value, and dividing the parameter difference value by the maximum parameter value to obtain the saturation parameter value. The saturation parameter value can be obtained by formula (1):

[0062] S = (C max -C min ) / C max formula (1);

[0063] wherein S is the saturation parameter value, C max is the maximum parameter value in the RGB color value, and C min is the minimum parameter value in the RGB color value.

[0064] 2. Hue parameter value H

[0065] ​The first channel parameter difference value is obtained by subtracting the parameter value corresponding to the blue color channel from the parameter value corresponding to the green color channel, and the first channel parameter difference value is divided by the parameter difference value and multiplied by the first specified parameter to obtain the hue parameter value. The hue parameter value can be obtained by formula (2):

[0066] H = [(C g -C b ) / (C max -C min )] x A formula (2);

[0067] wherein C g is the parameter value corresponding to the green color channel in the RGB color value, C b is the parameter value corresponding to the blue color channel in the RGB color value, and A is the first specified parameter. The first execution parameter in the embodiment is 60°.

[0068] The second channel parameter difference value is obtained by subtracting the parameter value corresponding to the red color channel from the parameter value corresponding to the blue color channel, the second channel parameter difference value is divided by the parameter difference value and multiplied by the first specified parameter to obtain a first intermediate value, and the first intermediate value is added to the second specified parameter to obtain the hue parameter value. The hue parameter value can be obtained by formula (3):

[0069] H = B + [(C b -C r ) / (C max -C min )] x A formula (3);

[0070] wherein B is the second specified parameter, the second specified parameter in the embodiment is 120°, and C r is the parameter value corresponding to the red color channel in the RGB color value.

[0071] The third channel parameter difference value is obtained by subtracting the parameter value corresponding to the green color channel from the parameter value corresponding to the blue color channel, the third channel parameter difference value is divided by the parameter difference value and multiplied by the first specified parameter to obtain a second intermediate value, and the second intermediate value is added to the third specified parameter to obtain the hue parameter value. The hue parameter value can be obtained by formula (4):

[0072] H = M + [(C r -C g ) / (C max -C min )] x A formula (4);

[0073] wherein, M is the third specified parameter, and the third specified parameter in the embodiment is 240°.

[0074] It should be noted that: if the hue parameter value is less than the second specified threshold, the hue parameter value is added to the fourth specified parameter to obtain the target hue parameter value.

[0075] wherein, the second specified threshold in the embodiment is 0°, and the fourth specified parameter value is 360°.

[0076] 3. The value of the luminance parameter V

[0077] The maximum parameter value in the RGB color value is determined as the value of the luminance parameter in the HSV color value, that is, max(C r , C g , C b ).

[0078] Step 2032: based on the HSV color value, obtaining the color three-dimensional space coordinates of each of the two feature points;

[0079] wherein, the HSV color space model in the embodiment is an HSV model cone with a hypotenuse length L, a bottom circle radius R, and a height h, and a color space coordinate system is established with the center of the bottom circle as the origin. The color three-dimensional space coordinates can be determined in the following manner:

[0080] 1. The horizontal coordinate x

[0081] The radius of the bottom circle of the HSV color space model, the value of the luminance parameter in the HSV color value, the value of the saturation parameter in the HSV color value, and the cosine value of the hue parameter in the HSV color value are multiplied to obtain the horizontal coordinate in the color three-dimensional space coordinates. The horizontal coordinate can be determined by formula (5):

[0082] x = R * V * S * cos H formula (5);

[0083] wherein, x is the horizontal coordinate, R is the radius of the bottom circle of the HSV color space model, V is the value of the luminance parameter in the HSV color value, S is the value of the saturation parameter in the HSV color value, and H is the value of the hue parameter in the HSV color value.

[0084] 2. The vertical coordinate y

[0085] The radius of the bottom circle of the HSV color space model, the value of the luminance parameter in the HSV color value, the value of the saturation parameter in the HSV color value, and the sine value of the hue parameter in the HSV color value are multiplied to obtain the vertical coordinate in the color three-dimensional space coordinates. The horizontal coordinate can be determined by formula (6):

[0086] y = R * V * S * sinH Equation (6);

[0087] wherein y is a vertical coordinate.

[0088] 3. Vertical coordinate

[0089] Subtracting the set value from the value of the lightness parameter in the HSV color value, a first difference value is obtained, and multiplying the first difference value by the height of the HSV color space model, the vertical coordinate is obtained. Wherein the vertical coordinate can be determined by Equation (7):

[0090] z = h * (1-V) Equation (7);

[0091] wherein z is a vertical coordinate, and h is the height of the HSV color space model.

[0092] Step 2033: obtaining the color similarity between the two feature points through the color three-dimensional space coordinates of the two feature points respectively.

[0093] It should be noted that: the values of the length of the hypotenuse L, the radius R of the base circle and the height h of the HSV color space model in the embodiment are all pre-set, which can be set according to actual conditions, and the embodiment does not limit here.

[0094] Step 204: if the color similarity of the two feature points is greater than a first specified threshold, it is determined that the two feature points are matched;

[0095] Step 205: if any two feature points with the same position coordinates in the two feature point sets are matched, it is determined that the poses of the two VR devices are the same, and the maps corresponding to the two VR devices respectively are fused based on the poses of the two VR devices.

[0096] For example, as shown in FIG. 1, image a is a map corresponding to VR device 1, and image b is a map corresponding to VR device 2, wherein m points in image a and image b are the same poses corresponding to the two VR devices, and the maps of the two VR devices are fused based on the m points, and the fused map is the map in image c. Figure 5

[0097] In order to ensure the accuracy of the map of the VR device, in one embodiment, after step 205 is performed, the fused map is sent to the two VR devices respectively, so that the two VR devices respectively update their own maps by using the fused map.

[0098] ​It should be noted that the manner of determining the pose based on the feature points in this embodiment can be set according to actual conditions, and the manner of map fusion in this embodiment can select the manner in the prior art, and this embodiment is not limited herein.

[0099] In order to further connect the technical solutions in the present application, the following will be described in detail in combination with Figure 6 may include the following steps:

[0100] Step 601: The server acquires target images respectively photographed by two VR devices;

[0101] Step 602: The server extracts a feature point set in each of the target images respectively;

[0102] Step 603: The server performs feature matching on the two feature point sets, and determines whether there is a target object with the same shape in the two target images based on the matching result, if yes, step 604 is performed, if not, the process is ended;

[0103] Step 604: The server extracts color information of each feature point in the corresponding feature point set from the two target images respectively;

[0104] The color information is an RGB color value.

[0105] Step 605: The server respectively converts the RGB color values of any two feature points with the same position coordinates in the two feature point sets into corresponding HSV color values; wherein the position coordinates are position coordinates in the same world coordinate system;

[0106] Step 606: The server obtains color three-dimensional space coordinates of the two feature points based on the HSV color values;

[0107] Step 607: The server obtains color similarity between the two feature points through the color three-dimensional space coordinates of the two feature points respectively;

[0108] Step 608: The server determines whether the two feature point sets are matched according to the color similarity of any two feature points with the same position coordinates in the two feature point sets, if yes, step 609 is performed, if not, the process is ended;

[0109] Step 609: The server fuses respective corresponding maps of the two VR devices based on poses of the two VR devices;

[0110] Step 610: The server sends the fused map to the two VR devices respectively;

[0111] Step 611: The two VR devices respectively update their own maps based on the fused map.

[0112] Based on the same inventive concept, the map fusion method of the VR device as described above can also be implemented by a map fusion apparatus of a VR device. The effect of the map fusion of the VR device is similar to that of the foregoing method, which will not be described here again.

[0113] Figure 7 A structural schematic diagram of the map fusion apparatus of the VR device according to an embodiment of the present disclosure.

[0114] As shown in Figure 7 The map fusion apparatus 700 of the VR device of the present disclosure can include a feature point set determination module 710, a feature matching module 720, a color similarity determination module 730, a feature point matching module 740, and a map fusion module 750.

[0115] The feature point set determination module 710 is configured to obtain target images respectively photographed by any two VR devices, and extract feature point sets in the target images, respectively.

[0116] The feature matching module 720 is configured to perform feature matching on the two feature point sets, and determine whether there is a target object with the same shape in the two target images based on the matching result.

[0117] The color similarity determination module 730 is configured to, if there is, extract color information of each feature point in the corresponding feature point set from the two target images, respectively, and determine color similarity between any two feature points with the same position coordinates in the two feature point sets based on the color information of the two feature points. The position coordinates are position coordinates in a same world coordinate system.

[0118] The feature point matching module 740 is configured to, if the color similarity between the two feature points is greater than a first specified threshold, determine that the two feature points are matched.

[0119] The map fusion module 750 is configured to, if any two feature points with the same position coordinates in the two feature point sets are matched, determine that the poses of the two VR devices are the same, and fuse the respective maps of the two VR devices based on the poses of the two VR devices.

[0120] In one embodiment, the color information is an RGB color value.

[0121] The color similarity determination module 730 is specifically configured to:

[0122] convert the respective RGB color values of the two feature points into corresponding HSV color values, respectively; and

[0123] obtain color three-dimensional space coordinates of the two feature points respectively based on the HSV color value;

[0124] obtain color similarity between the two feature points through the color three-dimensional space coordinates of the two feature points respectively.

[0125] In an embodiment, the color similarity determination module 730 is specifically configured to:

[0126] For any one feature point, the following steps are performed:

[0127] determine the maximum parameter value in the RGB color value as a lightness parameter value in the HSV color value; and,

[0128] obtain a saturation parameter value in the HSV color value according to the maximum parameter value and the minimum parameter value in the RGB color value; and,

[0129] obtain a hue parameter value in the HSV color value through the respective parameter values corresponding to the RGB color value;

[0130] obtain the HSV color value based on the lightness parameter value, the saturation parameter value and the hue parameter value.

[0131] In an embodiment, the color similarity determination module 730 is specifically configured to:

[0132] subtract the maximum parameter value from the minimum parameter value to obtain a parameter difference value, and divide the parameter difference value by the maximum parameter value to obtain the saturation parameter value;

[0133] The obtaining of the hue parameter value in the HSV color value through the respective parameter values corresponding to the RGB color value includes:

[0134] if the maximum parameter value in the RGB color value is a parameter value corresponding to a red channel, subtract a parameter value corresponding to a green channel from a parameter value corresponding to a blue channel to obtain a first channel parameter difference value, divide the first channel parameter difference value by the parameter difference value, multiply the result by a first specified parameter to obtain the hue parameter value; or,

[0135] if the maximum parameter value in the RGB color value is a parameter value corresponding to a green channel, subtract a parameter value corresponding to a blue channel from a parameter value corresponding to a red channel to obtain a second channel parameter difference value, divide the second channel parameter difference value by the parameter difference value, multiply the result by the first specified parameter to obtain a first intermediate value, add the first intermediate value to a second specified parameter to obtain the hue parameter value; or,

[0136] If the maximum parameter value in the RGB color value is the parameter value corresponding to the blue channel, the parameter value corresponding to the red channel is subtracted from the parameter value corresponding to the green channel to obtain a third channel parameter difference value, the third channel parameter difference value is divided by the parameter difference value, multiplied by the first specified parameter to obtain a second intermediate value, and the second intermediate value is added to the third specified parameter to obtain the hue parameter value.

[0137] In an embodiment, the apparatus further includes:

[0138] The hue parameter value adjustment module 760 is configured to, if the hue parameter value is less than a second specified threshold value, add the fourth specified parameter to the hue parameter value to obtain a target hue parameter value.

[0139] In an embodiment, the feature matching module 720 is specifically configured to:

[0140] For any feature point in any one of the two feature point sets, if the feature point has a feature point with the same position coordinates in the other feature point set, it is determined that the matching result is that the two feature point sets are matched.

[0141] Otherwise, it is determined that the matching result is that the two feature point sets are not matched.

[0142] In an embodiment, the apparatus further includes:

[0143] The sending module 770 is configured to, after the two maps corresponding to the two VR devices are fused based on the poses of the two VR devices, send the fused map to the two VR devices respectively, so that the two VR devices update the respective maps using the fused map.

[0144] After introducing the map fusion method and apparatus of a VR device according to an example embodiment of the present application, next, an electronic device according to another example embodiment of the present application is introduced.

[0145] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as follows: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0146] In some possible implementation manners, the electronic device according to the present application can at least include at least one processor and at least one computer storage medium. The computer storage medium stores program codes, when the program codes are executed by the processor, the processor executes the steps in the map fusion method of the VR device according to various exemplary embodiments of the present application described above in the specification. For example, the processor can execute the steps 201-203 as shown in Figure 2

[0147] The electronic device 800 according to this implementation manner of the present application will be described below with reference to Figure 8 Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the function and use range of the embodiments of the present application.

[0148] As shown in Figure 8 , the electronic device 800 is in the form of a general electronic device. The components of the electronic device 800 can include but are not limited to the at least one processor 801 described above, the at least one computer storage medium 802 described above, and the bus 803 connecting different system components including the computer storage medium 802 and the processor 801.

[0149] The bus 803 represents one or more of several types of bus structures, including a computer storage medium bus or a computer storage medium controller, a peripheral bus, a processor bus, or a local bus using any of a variety of bus structures.

[0150] The computer storage medium 802 can include readable media in the form of volatile computer storage medium, such as random access computer storage medium (RAM) 821 and / or cache storage medium 822, and can further include read-only computer storage medium (ROM) 823.

[0151] The computer storage medium 802 can further include program / utility 825 having a set of (at least one) program modules 824, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof can include implementation of a network environment.

[0152] ​​The electronic device 800 can also communicate with one or more external devices 804 such as a keyboard or a pointing device, through an input / output (I / O) interface 805. And, the electronic device 800 can communicate with one or more devices that enable user interaction with the electronic device 800, and / or one or more devices that enable communication of the electronic device 800 with one or more other electronic devices. This communication can be via the I / O interface 805. Further, the electronic device 800 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet) through a network adapter 806. As illustrated, the network adapter 806 communicates with the other modules of the electronic device 800 through the bus 803. It should be appreciated that although not shown, other hardware and / or software modules could be used in connection with the electronic device 800. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0153] In some possible embodiments, various aspects of the map fusion method of a VR device provided by the present application can also be implemented in the form of a program product, which includes program codes for causing a computer device to perform the steps of the map fusion method of a VR device according to various exemplary embodiments of the present application described above in the specification when the program product is run on the computer device.

[0154] The program product can employ any combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access computer memory (RAM), a read-only computer memory (ROM), an erasable programmable read-only computer memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only computer memory (CD-ROM), an optical computer storage device, a magnetic computer storage device, or any suitable combination of the above.

[0155] The program product of the map fusion of a VR device of the embodiments of the present application can employ a portable compact disc read-only computer memory (CD-ROM) and include program codes, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0156] A readable signal medium can be any available medium or media that can be accessed by a general purpose or special purpose electronic computer or other machine including a computer readable medium. By way of example, and not limitation, such computer readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium or media that can be used to carry or store desired computer readable code in the form of computer readable instructions, data structures, program modules or other data, alone or in combination with one another. In some embodiments, computer readable media can take the form of distributed network readable instructions, data structures, program modules or other data, which can be accessed by a general purpose or special purpose electronic computer or other machine for execution. In some embodiments, computer readable media can take the form of distributed network readable instructions, data structures, program modules or other data, which can be accessed by a general purpose or special purpose electronic computer or other machine for execution.

[0157] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0158] Program code, used by or in connection with the described embodiments, can be written in any of a number of suitable programming languages and / or programming or scripting tools, including one or more languages and / or tools suitable for creating one or more embodiments of the application, such as Java, C++, and / or other languages and / or tools. Such programming language will be

[0159] It should be noted that, although the above detailed description refers to several modules of the apparatus, such division is merely exemplary and not mandatory. Indeed, according to an embodiment of the application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into modules.

[0160] Moreover, while operations of the method of the application are described in a particular order in the drawings, this is not required or implied in any way as to the order of the operations or that all operations be performed to achieve desirable results. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or further divided into multiple steps.

[0161] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk computer storage media, CD-ROMs, optical computer storage media, etc.) containing computer-usable program code.

[0162] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable computer storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for map fusion of a VR device, the method comprising: The method comprises: obtaining target images respectively photographed by any two VR devices, and extracting feature point sets in the target images respectively; performing feature matching on the two feature point sets, and determining whether there is a target object with the same shape in the two target images based on the matching result; if there is, extracting color information of each feature point in the respective feature point sets from the two target images, and determining color similarity between any two feature points with the same position coordinates in the two feature point sets based on the color information of the two feature points; wherein the position coordinates are position coordinates in a same world coordinate system; and if the color similarity between the two feature points is greater than a first specified threshold, determining that the two feature points match; if any two feature points with the same position coordinates in the two feature point sets both match, determining that poses of the two VR devices are the same, and fusing respective maps of the two VR devices based on the poses of the two VR devices; wherein the color information is an RGB color value; the determining of the color similarity between the two feature points based on the color information of any two feature points with the same position coordinates in the two feature point sets comprises: converting the respective RGB color values of the two feature points into corresponding HSV color values; and obtaining horizontal and vertical coordinates in a color three-dimensional space coordinate based on the respective HSV color values of the two feature points and a radius of a bottom circle of an HSV color space model, obtaining a vertical coordinate in the color three-dimensional space coordinate based on the respective HSV color values of the two feature points and a height of the HSV color space model, and obtaining the respective color three-dimensional space coordinates of the two feature points based on the respective horizontal, vertical and vertical coordinates of the two feature points; obtaining the color similarity between the two feature points based on the respective color three-dimensional space coordinates of the two feature points. wherein the radius of the bottom circle of the HSV color space model and the height of the HSV color space model are pre-set.

2. The method of claim 1, wherein, the converting of the respective RGB color values of the two feature points into corresponding HSV color values comprises: for any one feature point, performing the following steps: determining a maximum parameter value in the RGB color value as a lightness parameter value in the HSV color value; and obtaining a saturation parameter value in the HSV color value according to the maximum parameter value and a minimum parameter value in the RGB color value; and obtaining a hue parameter value in the HSV color value based on the respective parameter values corresponding to the RGB color value; obtaining the HSV color value based on the lightness parameter value, the saturation parameter value and the hue parameter value.

3. The method according to claim 2, characterized in that the obtaining of the saturation parameter value in the HSV color value according to the maximum parameter value and the minimum parameter value in the RGB color value comprises: subtracting the minimum parameter value from the maximum parameter value to obtain a parameter difference value, and dividing the parameter difference value by the maximum parameter value to obtain the saturation parameter value. The hue parameter value in the HSV color value is obtained through the respective parameter values corresponding to the RGB color values, and includes: If the maximum parameter value in the RGB color value is the parameter value corresponding to the red channel, the parameter value corresponding to the green channel is subtracted from the parameter value corresponding to the blue channel to obtain a first channel parameter difference value, the first channel parameter difference value is divided by the parameter difference value, and then multiplied by a first specified parameter to obtain the hue parameter value; or, If the maximum parameter value in the RGB color value is the parameter value corresponding to the green channel, the parameter value corresponding to the blue channel is subtracted from the parameter value corresponding to the red channel to obtain a second channel parameter difference value, the second channel parameter difference value is divided by the parameter difference value, and then multiplied by the first specified parameter to obtain a first intermediate value, and the first intermediate value is added to a second specified parameter to obtain the hue parameter value; or, If the maximum parameter value in the RGB color value is the parameter value corresponding to the blue channel, the parameter value corresponding to the red channel is subtracted from the parameter value corresponding to the green channel to obtain a third channel parameter difference value, the third channel parameter difference value is divided by the parameter difference value, and then multiplied by the first specified parameter to obtain a second intermediate value, and the second intermediate value is added to a third specified parameter to obtain the hue parameter value.

4. The method according to claim 2 or 3, characterized in that, The method further includes: If the hue parameter value is less than a second specified threshold value, the hue parameter value is added to a fourth specified parameter to obtain a target hue parameter value.

5. The method of claim 1, wherein, The matching result is determined in the following manner: For any feature point in any one of the two feature point sets, if the feature point has a feature point with the same position coordinates in the other feature point set, it is determined that the matching result is that the two feature point sets match each other; otherwise, it is determined that the matching result is that the two feature point sets do not match each other. After the two maps corresponding to the two VR devices are fused based on the poses of the two VR devices, the method further includes:

6. The method of claim 1, wherein, The fused map is sent to the two VR devices respectively, so that the two VR devices update their respective maps using the fused map. The processor and the memory are connected through a bus; 7. An electronic device, comprising: The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: Obtain target images photographed by any two VR devices respectively, and extract feature point sets in the target images respectively; Perform feature matching on the two feature point sets, and determine whether there is a target object with the same shape in the two target images based on the matching result; If there is, extract color information of each feature point in the corresponding feature point set from the two target images, and determine color similarity between any two feature points with the same position coordinates in the two feature point sets; wherein the position coordinates are position coordinates in the same world coordinate system; and If the color similarity between the two feature points is greater than a first specified threshold value, it is determined that the two feature points match each other. ​ If any two feature points with the same position coordinates in the two sets of feature points are matched, it is determined that the poses of the two VR devices are the same, and the maps corresponding to the two VR devices respectively are fused based on the poses of the two VR devices. The color information is an RGB color value. The color similarity between the two feature points is determined based on the color information of any two feature points with the same position coordinates in the two sets of feature points, including: RGB color values of the two feature points are respectively converted into corresponding HSV color values; and Based on the HSV color values of the two feature points and the radius of the bottom circle of the HSV color space model, the horizontal coordinate and the vertical coordinate in the color three-dimensional space coordinate are obtained, based on the HSV color values of the two feature points and the height of the HSV color space model, the vertical coordinate in the color three-dimensional space coordinate is obtained, and based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the two feature points, the color three-dimensional space coordinates of the two feature points are obtained. The color similarity between the two feature points is obtained through the color three-dimensional space coordinates of the two feature points. The radius of the bottom circle of the HSV color space model and the height of the HSV color space model are pre-set. The color information is an RGB color value. The color similarity between the two feature points is determined based on the color information of any two feature points with the same position coordinates in the two sets of feature points, including: RGB color values of the two feature points are respectively converted into corresponding HSV color values; and Based on the HSV color values of the two feature points and the radius of the bottom circle of the HSV color space model, the horizontal coordinate and the vertical coordinate in the color three-dimensional space coordinate are obtained, based on the HSV color values of the two feature points and the height of the HSV color space model, the vertical coordinate in the color three-dimensional space coordinate is obtained, and based on the horizontal coordinate, the vertical coordinate and the vertical coordinate of the two feature points, the color three-dimensional space coordinates of the two feature points are obtained. The color similarity between the two feature points is obtained through the color three-dimensional space coordinates of the two feature points. The radius of the bottom circle of the HSV color space model and the height of the HSV color space model are pre-set.

Images