Image superimposition method and device of three-dimensional map, electronic equipment and storage medium
By establishing a projected coordinate system and performing inverse attenuation and blending calculations for distance attenuation, combined with anti-aliasing, the inconsistency problem when video textures and 3D maps are overlaid is solved, achieving a smooth transition effect and improving the coordination and accuracy of the overlay.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNISINSIGHT TECH CO LTD
- Filing Date
- 2022-11-25
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies often result in inconsistencies when overlaying video textures and 3D maps, with distinct transition regions. This is primarily due to the sampling accuracy and performance limitations of the Pencentage Close Filtering algorithm.
By using the field of view of the acquisition device as the projection object, a projection coordinate system is established. Through the reverse attenuation of the distance attenuation value and the calculation of the mixing value, a smooth transition of video images on the 3D map is achieved. Combined with anti-aliasing processing, the overlay effect is improved.
When overlaying video textures and 3D maps, a smooth transition effect is achieved in the transition area, improving the inconsistency problem and enhancing the coordination and accuracy of the overlay.
Smart Images
Figure CN115797579B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and more specifically, to a method, apparatus, electronic device, and storage medium for overlaying images of three-dimensional maps. Background Technology
[0002] With the widespread application of 3D maps and videos, in the process of video surveillance, real-time video images of the monitored area are usually overlaid onto a 3D map to observe and analyze the monitored area in combination with the environmental conditions of the periodic area.
[0003] Currently, in the process of overlaying real-time video images onto 3D maps, the classic real-time shadow generation algorithm (Shadow Mapping, SM) is mainly used to overlay video textures onto the 3D map, and the Pencentage Close Filtering algorithm is used to eliminate the obvious jagged edges generated at the boundary between the texture and the map during overlay. However, due to the sampling accuracy and performance consumption issues of Pencentage Close Filtering, although edge jagged edges are eliminated, the boundary between the video texture and the 3D map overlay is distinct, resulting in inconsistency. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and storage medium for overlaying three-dimensional maps, which can improve the inconsistency caused by the clear boundary between the transition area of video texture and three-dimensional map overlay.
[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:
[0006] In a first aspect, embodiments of the present invention provide an image overlay method for a three-dimensional map, the method comprising:
[0007] Using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, a projection coordinate system is established to obtain the projection area; wherein, the z-axis direction of the projection coordinate system is consistent with the line of sight of the acquisition device.
[0008] The image information acquired by the acquisition device is used as the overlay image, and the area to be overlaid in the three-dimensional map is determined.
[0009] For each point in the area to be superimposed, the distance attenuation value between the point and the center point of the projection area is obtained, and the distance attenuation value is attenuated in reverse to obtain the mixed value of the point.
[0010] The overlay image is superimposed onto the 3D map based on the blending value of each point.
[0011] Further, the step of overlaying the superimposed image onto the 3D map based on the blending value of each of the points includes:
[0012] For each point in the area to be overlaid, first image information corresponding to the point in the overlaid image and second image information corresponding to the point in the 3D map are obtained; wherein, both the first image information and the second image information include pixel values;
[0013] The mixing weight of the point is obtained based on the mixing value of the point; wherein the mixing weight includes the mixing weight of the first image information and the mixing weight of the second image information;
[0014] Based on the mixing weight of each point, the first image information, and the second image information, the mixed image information of each point in the region to be superimposed is obtained.
[0015] Further, the step of obtaining the distance attenuation value between each point in the area to be superimposed and the center point of the projection area includes:
[0016] Obtain the coordinate values of each point in the area to be superimposed in the projected coordinate system;
[0017] For each point, based on the coordinate value of the point, a first attenuation value between the point and the center point of the projection area in the y-axis direction of the projection coordinate system, and a second attenuation value between the point and the center point in the x-axis direction of the projection coordinate system are calculated.
[0018] The first attenuation value and the second attenuation value are compared, and the smaller value is determined as the distance attenuation value of the point.
[0019] Furthermore, prior to the step of overlaying the superimposed image onto the 3D map based on the blending value of each of the said points, the method further includes:
[0020] Anti-aliasing is applied to the overlay image to filter out jagged edges.
[0021] Further, the step of determining the area to be overlaid in the 3D map by the overlay image includes:
[0022] For each projection point in the projection area, multiple coordinate points whose projection positions are the projection points are determined from the coordinate points of the three-dimensional map;
[0023] From the multiple coordinate points of the projection point, the coordinate point closest to the projection point is taken as the target point;
[0024] The area formed by all the target points is taken as the area to be overlaid.
[0025] Further, the step of determining multiple coordinate points whose projection positions are the projection points from the coordinate points of the three-dimensional map for each projection point in the projection area includes:
[0026] Using the coordinates of the acquisition device on the 3D map as the viewpoint, an observation coordinate system is established, and a first depth map is generated using a frame buffer;
[0027] Using the observer's viewpoint on the 3D map as the viewpoint, an observation coordinate system is established, and a second depth map is generated using a frame buffer;
[0028] For each coordinate point in the 3D map, if the depth value of the coordinate point on the first depth map is greater than the depth value of the coordinate point on the second depth map, then the coordinate point is determined to be a coordinate point within the field of view of the acquisition device.
[0029] For each projection point in the projection area, the coordinates of the projection point are determined from all coordinate points within the field of view of the acquisition device.
[0030] Further, the step of inversely attenuating the distance attenuation value to obtain the mixed value of the point includes:
[0031] The distance attenuation value is subjected to a power function for inverse attenuation to obtain the mixed value of the point.
[0032] Secondly, embodiments of the present invention provide an image overlay device for a three-dimensional map, including a projection module, a first overlay module, and a second overlay module;
[0033] The projection module is used to establish a projection coordinate system and obtain the projection area by using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction; wherein, the z-axis direction of the projection coordinate system is consistent with the line of sight of the acquisition device.
[0034] The first overlay module is used to take the image information acquired by the acquisition device as an overlay image and determine the area to be overlaid in the three-dimensional map;
[0035] The second overlay module is used to obtain the distance attenuation value between each point in the area to be overlaid and the center point of the projection area, and to perform reverse attenuation on the distance attenuation value to obtain the mixed value of the point.
[0036] The second overlay module is further configured to overlay the overlay image onto the three-dimensional map based on the mixing value of each of the points.
[0037] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the image overlay method for three-dimensional maps as described in the first aspect.
[0038] Fourthly, embodiments of the present invention provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the image overlay method for three-dimensional maps as described in the first aspect.
[0039] The three-dimensional map image overlay method, apparatus, electronic device, and storage medium provided in this invention establish a projection coordinate system by using the coordinates of the acquisition device as the projection center point and the orientation of the acquisition device as the projection direction. This obtains the projection area of the acquisition device's field of view and uses the image information acquired by the acquisition device as the overlay image. The overlay image is then determined in the area to be overlaid on the three-dimensional map. The distance attenuation value of each point in the area to be overlaid is then attenuated in reverse to approximate the attenuation area to the edge of the area to be overlaid, obtaining the blending value of each point. Based on the blending value of each point, the overlay image is overlaid onto the three-dimensional map. Thus, based on the attenuation of each point relative to the projection center point, the overlay image and the three-dimensional map are overlaid in the area to be overlaid, achieving a smooth transition effect at the edge of the area to be overlaid. This greatly improves the inconsistency caused by the sharp boundaries of the transition area.
[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 A block diagram of an image overlay system provided in an embodiment of the present invention is shown.
[0043] Figure 2 This is a schematic flowchart of an image overlay method for three-dimensional maps provided in an embodiment of the present invention.
[0044] Figure 3 A schematic diagram of the projection coordinate system provided in an embodiment of the present invention is shown.
[0045] Figure 4 It shows Figure 2 A flowchart illustrating some sub-steps of step S13.
[0046] Figure 5 It shows Figure 4 A flowchart illustrating some sub-steps of step S131.
[0047] Figure 6 It shows Figure 2 A flowchart illustrating some sub-steps of step S15.
[0048] Figure 7 This is a second schematic flowchart of the image overlay method for three-dimensional maps provided in an embodiment of the present invention.
[0049] Figure 8 It shows Figure 7 A flowchart illustrating some sub-steps of step S17.
[0050] Figure 9 A block diagram of an image overlay device for a three-dimensional map provided in an embodiment of the present invention is shown.
[0051] Figure 10 A block diagram of an electronic device provided in an embodiment of the present invention is shown.
[0052] Reference numerals: 100-Image overlay system; 110-Acquisition device; 120-Server; 130-Display terminal; 140-Image overlay device for 3D map; 150-Projection module; 160-First overlay module; 170-Second overlay module; 180-Electronic device. Detailed Implementation
[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0054] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0055] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0056] Currently, in the process of overlaying real-time video images onto 3D maps, the classic real-time shadow generation algorithm (Shadow Mapping, SM) is mainly used to overlay video textures onto 3D maps, and the PencentageClose Filtering (PCF) algorithm is used to eliminate obvious jagged edges generated at the junction of texture edges and map during overlay.
[0057] Since the shadow result obtained by the above SM algorithm is a visibility value that is either zero or one (visibility of 1 means visible, and visibility of 0 means invisible), a harsh transition effect, i.e., a jagged effect, will be generated at the edge of the shadow area.
[0058] Due to the sampling accuracy and performance consumption issues of Pencentage Close Filtering, although it can eliminate edge jaggedness, the boundary between the video texture and the 3D map overlay is distinct, resulting in inconsistency.
[0059] Based on the above considerations, embodiments of the present invention provide an image overlay method for three-dimensional maps, which can improve the inconsistency problem caused by the distinct boundary between the transition area of video texture and three-dimensional map overlay.
[0060] The image overlay method for three-dimensional maps provided in this embodiment of the invention can be applied to, for example... Figure 1The image overlay system 100 shown includes an acquisition device 110, a server 120, and a display terminal 130. The server 120 can communicate with the acquisition device 110 and the display terminal 130 via a network.
[0061] The acquisition device 110 is used to acquire image information of the monitored area and send the image information to the server 120.
[0062] Server 120 is used to receive image information and implement the image overlay method for three-dimensional maps provided in this embodiment of the invention, so as to overlay the image information onto the three-dimensional map to obtain the overlaid three-dimensional map, and send the overlaid three-dimensional map to the display terminal 130.
[0063] Display terminal 130 is used to receive and display the overlaid 3D map.
[0064] Display device 130 may be, but is not limited to: wearable mobile devices, mobile phones, laptops, personal computers, and iPads, etc.
[0065] In one possible implementation, refer to Figure 2 This invention provides a method for overlaying images of a three-dimensional map, which may include the following steps. In this embodiment, the method for overlaying images of a three-dimensional map is applied to... Figure 1 Let's take server 120 as an example.
[0066] S11. Using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, a projection coordinate system is established, and the projection area is obtained.
[0067] Reference Figure 3 The z-axis of the projected coordinate system (P-xyz) can be aligned with the line-of-sight direction (line-of-sight center line) of the acquisition device 110. The origin of the projected coordinate system is the coordinate point of the acquisition device 110, and its x-axis and y-axis are perpendicular to each other on the horizontal plane.
[0068] S13, use the image information acquired by the acquisition device as the overlay image, and determine the area to be overlaid on the 3D map.
[0069] S15, for each point in the area to be superimposed, obtain the distance attenuation value between the point and the center point of the projection area, and perform reverse attenuation on the distance attenuation value to obtain the mixed value of the point.
[0070] In one possible implementation, to make the attenuation region as close as possible to the edge region of the area to be overlaid, the distance attenuation value is attenuated in reverse using a power function to obtain the blended value of the point. Thus, the attenuation value only affects the edge transparency of the area to be overlaid.
[0071] S17: Based on the blending value of each point, overlay the image onto the 3D map.
[0072] It should be noted that the monitoring area of the acquisition device 110 is a region within the scene corresponding to the 3D map; therefore, the image information can also be understood as a real-time image of a certain region on the 3D map. Furthermore, the projection area is a surface, which can coincide with the ground of the monitored area in the 3D map.
[0073] The acquisition device 110 transmits the image information of the monitored area to the server 120 in real time. The server 120 obtains the GPS spatial coordinates and XYZ three-axis attitude coordinates of the acquisition device 110 and receives the image information transmitted by the acquisition device 110 in real time. The server 120 establishes a projection coordinate system (P-xyz coordinate system) with the coordinates of the acquisition device 110 on the 3D map (i.e., GPS spatial coordinates) as the projection center point (point P) and the camera orientation of the acquisition device 110 as the projection direction (in this embodiment, the z-axis direction). The server then projects the field of view of the acquisition device 110 to determine the projection area of the field of view of the acquisition device 110 on the 3D map.
[0074] Server 120 uses the real-time received image information as an overlay image to determine the area to be overlaid on the 3D map. It should be understood that the area to be overlaid is the monitoring area of acquisition device 110, meaning that the overlay image and the area to be overlaid belong to the same area in the scene corresponding to the 3D map.
[0075] After determining the area to be overlaid, server 120 obtains the distance attenuation value between each point in the area to be overlaid and the center point of the projection area, and performs reverse attenuation on the distance attenuation value to make the attenuation area approximate the edge map of the area to be overlaid, thus obtaining the blending value of each point. Then, based on the blending value of each point, the overlay image is superimposed onto the 3D map in the area to be overlaid.
[0076] Compared with traditional image texture and 3D map overlay methods, the 3D map image overlay method provided in this embodiment of the invention, after acquiring the overlay image and determining the area to be overlaid and the projection area of the field of view of the acquisition device, overlays the image and the 3D map in the area to be overlaid according to the attenuation of each point and the projection center point, and achieves a smooth transition effect at the edge of the area to be overlaid, which can greatly improve the incoordination caused by the distinct boundary of the transition area.
[0077] The method for determining the overlay area of the image on the 3D map can be flexibly set. For example, it can be determined according to preset rules or through a neural network. In this embodiment, no specific limitation is made.
[0078] To more accurately determine the area to be overlaid on the 3D map, in one possible implementation, refer to... Figure 4 This can be further implemented as follows:
[0079] S131, for each projection point in the projection area, determine multiple coordinate points whose projection location is the projection point from the coordinate points of the three-dimensional map.
[0080] S132: From the multiple coordinate points of the projection point, select the coordinate point closest to the projection point as the target point.
[0081] S133, the area formed by all target points is taken as the area to be overlaid.
[0082] The 3D map corresponds to a 3D scene, while the field of view of the acquisition device 110 is a frustum (see reference). Figure 3 Therefore, the projection positions of multiple coordinate points within the field of view of the acquisition device 110 and multiple coordinate points outside the field of view in the 3D map may all be the same projection point in the projection area, which will interfere with the definition of the projection area and affect the accuracy of the projection area.
[0083] Considering the above problems, in order to improve the accuracy of the projection area, refer to Figure 5 The above step S131 can be further implemented by the following steps.
[0084] S1311 uses the coordinates of the acquisition device on the 3D map as the viewpoint to establish an observation coordinate system and uses the frame buffer to generate the first depth map.
[0085] It should be understood that the 3D map observed in the coordinate system corresponds to the actual scene.
[0086] S1312 establishes an observation coordinate system using the observer's viewpoint on the 3D map as the viewpoint, and generates a second depth map using the frame buffer.
[0087] The observation point can also be understood as the observer's focal point, and the observer's object of observation is the 3D map. The 3D map in S1312 is the rendered map.
[0088] S1313: For each coordinate point in the 3D map, if the depth value of the coordinate point on the first depth map is greater than the depth value of the coordinate point on the second depth map, then the coordinate point is determined to be a coordinate point within the field of view of the acquisition device.
[0089] For example, for each coordinate point in the 3D map, if the depth value of the coordinate point on the first depth map is greater than the depth value of the coordinate point on the second depth map, then the visibility value of the coordinate point is: visibility = 1; otherwise, the visibility value is: visibility = 0. visibility = 1 indicates that it is within the visible range of the acquisition device 110, and visibility = 0 indicates that it is outside the visible range of the acquisition device 110.
[0090] S1314, For each projection point in the projection area, determine the coordinates of the projection point from all coordinate points within the field of view of the acquisition device.
[0091] After determining all coordinate points within the visible range of the acquisition device 110, the coordinate points corresponding to each projection point within the projection area are determined from these coordinate points, i.e., the coordinate points of the projection point are the projection positions.
[0092] By using the above steps S131-S133 and the sub-steps of S131, the region to be superimposed corresponding to the superimposed image can be determined more accurately.
[0093] It should be understood that for the same acquisition device 110 in the same scene, the area to be overlaid only needs to be determined once.
[0094] The method for obtaining the distance attenuation value between the point and the center point of the projection area can be flexibly set. For example, the distance attenuation value between the point and the center point of the projection area can be directly calculated, or a machine learning algorithm can be used to fit the distance attenuation value. In this embodiment, no specific limitation is made.
[0095] To ensure that the distance attenuation value more fully describes the distance attenuation between each point in the area to be superimposed and the center point of the projected area, thereby improving the coordination of the superimposition, in one possible implementation, reference is made to... Figure 6 The method for obtaining the distance attenuation value between the point and the center point of the projection area can be implemented as follows:
[0096] S151, obtain the coordinate values of each point in the area to be overlaid in the projected coordinate system.
[0097] S152, for each point, calculate the first attenuation value between the point and the center point of the projected area in the y-axis direction of the projected coordinate system, and the second attenuation value between the point and the center point in the x-axis direction of the projected coordinate system, based on the coordinate value of the point.
[0098] S153, compare the first attenuation value and the second attenuation value, and determine the smaller value as the distance attenuation value of the point.
[0099] Through the above steps S151-S153, the distance attenuation value between the point and the center point of the projection area is calculated from the x-axis and y-axis respectively, and the smaller value is selected as the distance attenuation value of the point. In this way, the attenuation area can be brought closer to the edge area with the mixing value corresponding to the smaller distance attenuation value of each point, so that the distance attenuation value only affects the edge transparency of the area to be superimposed, thereby improving the superimposition effect.
[0100] In one possible implementation, in order to make the attenuation region as close as possible to the edge region of the region to be superimposed, for step S15, the mixed value of the point can be obtained by performing a power function on the distance attenuation value to attenuate it in reverse, thereby obtaining the mixed value of the point.
[0101] The power function can be: f(x) = e -10x , where f(x) can represent the mixed value and x represents the decay value.
[0102] Using a power function for inverse attenuation ensures that the attenuation value only affects the edge transparency of the area to be overlaid, which helps to improve the effect of a smooth transition.
[0103] To avoid jagged edges in the superimposed area after image overlay, in one possible implementation, refer to Figure 7 The image overlay method for three-dimensional maps provided in this embodiment of the invention may further include step S16. Step S16 is executed before step S17, and may be executed simultaneously with step S15, or sequentially.
[0104] S16, perform anti-aliasing processing on the superimposed image to filter out jagged edges of the superimposed image.
[0105] The method for anti-aliasing the superimposed image can be flexibly selected. For example, it can be processed according to preset rules or any anti-aliasing algorithm can be used. No specific limitation is made in this embodiment.
[0106] In this embodiment, the PCF algorithm is used to process the edge points of the overlaid image.
[0107] In one possible implementation, refer to Figure 8 Step S17 can be further implemented as follows.
[0108] S171, for each point in the area to be overlaid, obtain the first image information corresponding to the point in the overlaid image and the second image information corresponding to the point in the three-dimensional map.
[0109] Both the first image information and the second image information include pixel values.
[0110] S172. Obtain the mixing weights of the points according to the mixing values of the points. Among them, the mixing weights include the mixing weights of the first image information and the mixing weights of the second image information.
[0111] S173. Obtain the mixed image information of each point in the area to be superimposed according to the mixing weights of each point, the first image information, and the second image information.
[0112] The mixing value is represented by d. For example, if the mixing value of a certain point is d = 1, then the mixing weight of the first image information of this point is 1, and the mixing weight of the second image information is 0. When superimposing, the image data (i.e., the first image information) at the position corresponding to this point in the superimposed image is used as the image data of this point after superimposition.
[0113] If the mixing value of a certain point is d = 0, then the mixing weight of the first image information of this point is 0, and the mixing weight of the second image information is 1. When superimposing, the image data (i.e., the second image information) of this point in the three-dimensional map is used as the image data of this point after superimposition.
[0114] If the mixing value of a certain point is d = 0.2, 0 < d < 1, at this time, the mixing weight of the first image information of this point is 0.2, and the mixing weight of the second image information is 0.8. When superimposing, the sum of the product of the image data (i.e., the first image information) at the position corresponding to this point in the superimposed image and 0.2, and the product of the image data (i.e., the second image information) of this point in the three-dimensional map and 0.8 is used as the image data of this point after superimposition.
[0115] The image data of each point in the area to be superimposed after superimposition can be expressed as: P = tp1×u1 + tp2×u2, where P represents the image data of the point, tp1 represents the first image information of the point, u1 represents the mixing weight of the first image information, tp2 represents the second image information of the point, and u2 represents the mixing weight of the second image information.
[0116] The ideological principle of the above steps S171 - S173 is: For an image, the pixels closest to the boundary of light and shadow are surrounded by pixels with a shadow mapping value less than 0.5 and pixels with a shadow mapping value greater than or equal to 0.5. If these adjacent pixels are sampled and the results are averaged, a factor (visibility) that can help smooth the boundary of light and shadow will be obtained. The value of the factor is not simply a non - zero or one value, so as to produce a transitional effect at the boundary.
[0117] Through the above steps S171 - S173, when performing the superimposition, fully consider the attenuation situation of each point in the area to be superimposed and the projection center point of the projection center, so as to achieve a soft transition at the edge of the superimposed area.
[0118] Based on the inventive concept of the three-dimensional map image overlay method provided in the above embodiments, in one possible embodiment, the present invention also provides a three-dimensional map image overlay device 140. In this embodiment, the device can also be applied to... Figure 1 Server 120 in the middle. (Refer to...) Figure 9 The image overlay device 140 of the three-dimensional map may include a projection module 150, a first overlay module 160, and a second overlay module 170.
[0119] The projection module 150 is used to establish a projection coordinate system by using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, and to obtain the projection area. The z-axis of the projection coordinate system is aligned with the line of sight of the acquisition device.
[0120] The first overlay module 160 is used to use the image information acquired by the acquisition device as an overlay image and to determine the area to be overlaid in the three-dimensional map.
[0121] The second overlay module 170 is used to obtain the distance attenuation value between each point in the area to be overlaid and the center point of the projection area, and to perform reverse attenuation on the distance attenuation value to obtain the mixed value of the point.
[0122] The second overlay module 170 is also used to overlay the image onto the 3D map based on the blending value of each point.
[0123] In the aforementioned three-dimensional map image overlay device 140, through the coordinated action of the projection module 150, the first overlay module 160, and the second overlay module 170, after acquiring the overlay image and determining the area to be overlaid and the projection area of the field of view of the acquisition device, the overlay image and the three-dimensional map are overlaid in the area to be overlaid according to the attenuation of each point and the projection center point, achieving a smooth transition effect at the edge of the area to be overlaid, which can greatly improve the incoordination caused by the distinct boundaries of the transition area.
[0124] Specific limitations regarding the image overlay device 140 for 3D maps can be found in the above description of the image overlay method and index demodulation method for 3D maps, and will not be repeated here. Each module in the aforementioned image overlay device 140 for 3D maps can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in an electronic device, or stored in software in the memory of the electronic device, so that the processor can call and execute the corresponding operations of each module.
[0125] In one embodiment, an electronic device 180 is provided, which may be a server 120, and its internal structure diagram may be as follows: Figure 10 As shown, the electronic device 180 includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor of the electronic device 180 provides computing and control capabilities. The memory of the electronic device 180 includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device 180 is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, near-field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements the image overlay method for three-dimensional maps as described in the above embodiments.
[0126] Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 180 to which the present invention is applied. The specific electronic device 180 may include, but is not limited to, the following: Figure 10 The diagram shows more or fewer components, or combinations of certain components, or different component arrangements.
[0127] In one embodiment, the three-dimensional map image overlay device 140 provided by the present invention can be implemented as a computer program, which can be implemented in, for example... Figure 10 The electronic device 180 shown operates on this device. The memory of the electronic device 180 can store the various program modules of the image overlay device 140 that constitute the three-dimensional map, for example, Figure 9 The projection module 150, the first overlay module 160, and the second overlay module 170 are shown. The computer program, composed of these various program modules, causes the processor to execute the steps in the image overlay method for three-dimensional maps described in this specification.
[0128] For example, Figure 10 The electronic device 180 shown can be accessed via, for example Figure 9 The projection module 150 in the image overlay device 140 of the shown 3D map executes step S11. The electronic device 180 can execute step S13 through the first overlay module 160. The electronic device 180 can execute steps S15-S17 through the second overlay module 170.
[0129] In one embodiment, an electronic device 180 is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, a projection coordinate system is established, and a projection area is obtained; the image information acquired by the acquisition device is used as a superimposed image, and the superimposed image is determined in the superimposed area of the 3D map; for each point in the superimposed area, the distance attenuation value between the point and the center point of the projection area is obtained, and the distance attenuation value is attenuated in reverse to obtain a blending value for the point; based on the blending value of each point, the superimposed image is superimposed onto the 3D map.
[0130] In one embodiment, a storage medium is provided storing a computer program that, when executed by a processor, performs the following steps: establishing a projection coordinate system using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, and obtaining the projection area; using the image information acquired by the acquisition device as a superimposed image, and determining the superimposed area of the superimposed image on the 3D map; for each point in the superimposed area, obtaining the distance attenuation value between the point and the center point of the projection area, performing reverse attenuation on the distance attenuation value to obtain the mixing value of the point; and superimposing the superimposed image onto the 3D map according to the mixing value of each point.
[0131] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0132] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0133] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for overlaying images in a three-dimensional map, characterized in that, The method includes: Using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction, a projection coordinate system is established to obtain the projection area; wherein, the z-axis direction of the projection coordinate system is consistent with the line of sight of the acquisition device. The image information acquired by the acquisition device is used as the overlay image, and the area to be overlaid in the three-dimensional map is determined. For each point in the area to be superimposed, the distance attenuation value between the point and the center point of the projection area is obtained, and the distance attenuation value is attenuated in reverse to obtain the mixed value of the point. The step of obtaining the distance attenuation value between each point in the area to be superimposed and the center point of the projection area includes: Obtain the coordinate values of each point in the area to be superimposed in the projected coordinate system; For each point, based on the coordinate value of the point, a first attenuation value between the point and the center point of the projection area in the y-axis direction of the projection coordinate system, and a second attenuation value between the point and the center point in the x-axis direction of the projection coordinate system are calculated. The first attenuation value and the second attenuation value are compared, and the smaller value is determined as the distance attenuation value of the point. The overlay image is superimposed onto the 3D map based on the blending value of each point; The step of overlaying the image onto the 3D map based on the blending value of each point includes: For each point in the area to be overlaid, first image information corresponding to the point in the overlaid image and second image information corresponding to the point in the 3D map are obtained; wherein, both the first image information and the second image information include pixel values; The mixing weight of the point is obtained based on the mixing value of the point; wherein the mixing weight includes the mixing weight of the first image information and the mixing weight of the second image information; Based on the mixing weight of each point, the first image information, and the second image information, the mixed image information of each point in the region to be superimposed is obtained.
2. The image overlay method for three-dimensional maps according to claim 1, characterized in that, Prior to the step of overlaying the image onto the 3D map based on the blending value of each of the said points, the method further includes: Anti-aliasing is applied to the overlay image to filter out jagged edges.
3. The image overlay method for three-dimensional maps according to claim 1, characterized in that, The step of determining the area to be overlaid in the 3D map includes: For each projection point in the projection area, multiple coordinate points whose projection positions are the projection points are determined from the coordinate points of the three-dimensional map; From the multiple coordinate points of the projection point, the coordinate point closest to the projection point is taken as the target point; The area formed by all the target points is taken as the area to be overlaid.
4. The image overlay method for three-dimensional maps according to claim 3, characterized in that, The step of determining multiple coordinate points whose projection location is the projection point from the coordinate points of the 3D map for each projection point in the projection area includes: Using the coordinates of the acquisition device on the 3D map as the viewpoint, an observation coordinate system is established, and a first depth map is generated using a frame buffer; Using the observer's viewpoint on the 3D map as the viewpoint, an observation coordinate system is established, and a second depth map is generated using a frame buffer; For each coordinate point in the 3D map, if the depth value of the coordinate point on the first depth map is greater than the depth value of the coordinate point on the second depth map, then the coordinate point is determined to be a coordinate point within the field of view of the acquisition device. For each projection point in the projection area, the coordinates of the projection point are determined from all coordinate points within the field of view of the acquisition device.
5. The image overlay method for three-dimensional maps according to claim 1, characterized in that, The step of inversely attenuating the distance attenuation value to obtain the mixed value of the point includes: The distance attenuation value is subjected to a power function for inverse attenuation to obtain the mixed value of the point.
6. A three-dimensional map image overlay device, characterized in that, It includes a projection module, a first overlay module, and a second overlay module; The projection module is used to establish a projection coordinate system and obtain the projection area by using the field of view of the acquisition device as the projection object, the coordinates of the acquisition device on the 3D map as the projection center point, and the orientation of the acquisition device as the projection direction; wherein, the z-axis direction of the projection coordinate system is consistent with the line of sight of the acquisition device. The first overlay module is used to take the image information acquired by the acquisition device as an overlay image and determine the area to be overlaid in the three-dimensional map; The second overlay module is used to obtain the distance attenuation value between each point in the area to be overlaid and the center point of the projection area, and to perform reverse attenuation on the distance attenuation value to obtain the mixed value of the point. The second overlay module is further configured to obtain the coordinate values of each point in the area to be overlaid in the projection coordinate system; for each point, based on the coordinate values of the point, calculate a first attenuation value between the point and the center point of the projection area in the y-axis direction of the projection coordinate system, and a second attenuation value between the point and the center point in the x-axis direction of the projection coordinate system; compare the first attenuation value and the second attenuation value, and determine the smaller value as the distance attenuation value of the point; The second overlay module is further configured to overlay the overlay image onto the three-dimensional map based on the mixing value of each of the points; The second overlay module is further configured to, for each point in the area to be overlaid, acquire first image information corresponding to the point in the overlaid image and second image information corresponding to the point in the three-dimensional map; wherein, both the first image information and the second image information include pixel values; obtain the mixing weight of the point based on the mixing value of the point; wherein, the mixing weight includes the mixing weight of the first image information and the mixing weight of the second image information; and obtain the mixed image information of each point in the area to be overlaid based on the mixing weight of each point, the first image information, and the second image information.
7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor to implement the image overlay method for a three-dimensional map as described in any one of claims 1 to 6.
8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image overlay method for three-dimensional maps as described in any one of claims 1 to 6.