An image reflection elimination method and system based on panoramic camera

Abstract

The application is suitable for the technical field of image reflection elimination, and particularly relates to an image reflection elimination method and system based on panoramic imaging, which comprises the following steps: collecting panoramic images in a use area by using a panoramic imaging device; irradiating the use area according to a traversing rule which has been edited, by using an infrared emitter which is pre-installed in the use area; and recording the spatial projection track of the infrared light points emitted by the infrared emitter in the use area; and using an image sensing device which is installed in the use area. By combining semantic description and image repair model, the application can complete the compensation of the image reflection area under the premise of ensuring semantic consistency, avoid content mismatch, realize automatic optimization processing of panoramic images, improve the detail optimization level and boundary smoothing processing efficiency of the compensation area, greatly enhance the visual consistency of panoramic images, and further improve the generalization ability of the panoramic imaging device under different time periods and environments.

Description

Technical Field

[0001] This invention relates to the field of image reflection elimination technology, and in particular to an image reflection elimination method and system based on panoramic photography. Background Technology

[0002] Panoramic images offer advantages such as a wide field of view and comprehensive coverage, making them widely used in fields such as security monitoring, intelligent transportation, robot navigation, and virtual reality. However, in practical applications, the presence of highly reflective media such as glass curtain walls, metal surfaces, and water surfaces in the shooting environment can easily cause significant reflections in panoramic images, resulting in issues such as bright spots or localized overexposure, which severely affect image quality.

[0003] Existing technologies primarily address image reflection issues through image enhancement and simple image restoration. However, neither method fundamentally eliminates reflection information or handles large reflective areas, often resulting in image distortion. Introducing other cameras to capture images of the same area from multiple perspectives, and utilizing the imaging differences between these perspectives to calibrate the reflective areas, can fundamentally compensate for the information loss from a single perspective.

[0004] Therefore, "how to use other cameras to compensate for information in the image reflection area" is the technical problem that this invention needs to solve. Summary of the Invention

[0005] The purpose of this invention is to provide an image reflection elimination method and system based on panoramic photography, so as to solve the problem of "how to use other cameras to compensate for information in the image reflection area" mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for eliminating image reflections based on panoramic photography, the method comprising:

[0008] Using panoramic camera equipment, panoramic images of the area to be used are captured. The area to be used is then illuminated by an infrared transmitter pre-installed in the area according to a pre-edited traversal rule. The spatial projection trajectory of the infrared light spot emitted by the infrared transmitter within the area to be used is recorded.

[0009] Using image sensing devices installed in the usage area, the movement data of infrared light spots are collected, and timestamps are embedded into the panoramic camera and image sensing devices to align the spatial projection trajectory and movement data.

[0010] The image reflection areas in the panoramic image are identified, boundary lines are delineated, and defined as the movement path of the infrared light points. The movement path is sent to the infrared transmitter. Based on the image sensing device, real-time video data in the area is collected, the part composed of the movement path is extracted and defined as the reference area, and a semantic description is generated. The image reflection areas are masked, and the masked panoramic image and semantic description are input into a pre-trained image restoration model to output a complete image. The complete image is then sent to a preset terminal to update the panoramic image.

[0011] Furthermore, the step of using a panoramic camera device to acquire panoramic images of the usage area, and then transmitting them via an infrared transmitter pre-installed in the usage area according to pre-edited traversal rules includes:

[0012] Collect environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution;

[0013] Based on the environmental data, several marker points are selected within the usage area, and the installation location of each marker point is set.

[0014] Furthermore, the method also includes:

[0015] The panoramic image is divided into several sub-regions, and the probability level of the occurrence of image reflection areas in each sub-region is determined;

[0016] The factors influencing the probability level of occurrence are set, including at least: light, time of day, and spatial structure, and emergency response rules are created corresponding to each factor.

[0017] Furthermore, the step of acquiring movement data of infrared light spots using image sensing devices installed within the usage area includes:

[0018] Build a management platform for image sensing devices, grant registration permissions to the management platform, and identify registered users;

[0019] The system receives image data uploaded by registered users, obtains sample images, and establishes a correspondence between the sample images and the image reflection areas.

[0020] Furthermore, the step of extracting the portion composed of the movement path and defining it as a reference region to generate a semantic description includes:

[0021] Create a semantic parsing model, randomly select several sub-regions and define them as sample regions, manually annotate the sample regions, generate a training set, and train the semantic parsing model;

[0022] The reference region is input into the trained semantic parsing model, and the output is a semantic description.

[0023] Furthermore, the step of inputting the masked panoramic image and semantic description into a pre-trained image inpainting model, outputting a complete image, and sending the complete image to a preset terminal to update the panoramic image includes:

[0024] According to a preset time step, a number of video frames are extracted from the image data captured by the panoramic camera device and defined as a panoramic image;

[0025] A panoramic image containing image reflection areas is defined as a defective image, and the complete image is used to replace the defective image to correct the panoramic image.

[0026] Furthermore, the system includes:

[0027] The recording module is used to acquire panoramic images of the area under use using a panoramic camera device, and illuminate the area under use according to the pre-edited traversal rules via an infrared emitter pre-installed in the area under use, and record the spatial projection trajectory of the infrared light spot emitted by the infrared emitter within the area under use.

[0028] The alignment module is used to collect the movement data of infrared light spots using the image sensing device installed in the area of ​​use, embed timestamps into the panoramic camera and the image sensing device, and align the spatial projection trajectory and movement data.

[0029] The update module is used to identify image reflection areas in the panoramic image, delineate boundary lines, and define them as the movement path of infrared light points. The movement path is sent to the infrared transmitter. Based on the image sensing device, real-time video data in the area is collected, the part composed of the movement path is extracted and defined as the reference area, a semantic description is generated, the image reflection areas are masked, the masked panoramic image and semantic description are input into a pre-trained image restoration model, a complete image is output, and the complete image is sent to a preset terminal to update the panoramic image.

[0030] The segmentation module is used to segment the panoramic image into several sub-regions, determine the probability level of the occurrence of image reflection areas in each sub-region, set the influencing factors of the occurrence probability level, wherein the influencing factors include at least: light, time of day and spatial structure, and create emergency response rules corresponding to each influencing factor.

[0031] Furthermore, the recording module includes:

[0032] A data acquisition unit is used to collect environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution;

[0033] The setting unit is used to select several marker points within the usage area based on the environmental data, and set the installation location of each marker point.

[0034] Furthermore, the alignment module includes:

[0035] The building unit is used to build a management platform for image sensing devices, open up registration permissions for the management platform, and identify registered users;

[0036] The receiving unit is used to receive image data uploaded by registered users, obtain sample images, and establish the correspondence between sample images and image reflection areas.

[0037] Furthermore, the update module includes:

[0038] A creation unit is used to create a semantic parsing model. Several sub-regions are randomly selected and defined as sample regions. The sample regions are manually labeled to generate a training set for training the semantic parsing model.

[0039] The input unit is used to input the reference region into the trained semantic parsing model and output a semantic description.

[0040] The cropping unit is used to extract a number of video frames from the image data captured by the panoramic camera device according to a preset time step, and define them as a panoramic image;

[0041] The correction unit is used to define a panoramic image with an image reflection area as a defective image, replace the defective image with the complete image, and correct the panoramic image.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] By utilizing image sensing devices to collect the movement data of infrared light spots, reflection areas can be delineated, providing positional references for panoramic image completion and compensating for panoramic image defects. Simultaneously, by matching the movement trajectories of the infrared light spots, a mapping between the panoramic image and the image sensing device's capture data can be established as spatial anchor points, constructing a unified spatial reference. By determining the reference area, the corresponding part of the image reflection area can be found in the image sensing device's capture data, achieving spatiotemporal alignment of multi-source image data. By generating semantic descriptions, accurate identification of the content of the image reflection area can be achieved, improving the accuracy of compensation and providing a data foundation for information compensation of the image reflection area. By combining semantic descriptions and image restoration models, image reflection area compensation can be completed while ensuring semantic consistency, avoiding content mismatches and achieving automatic optimization processing of panoramic images. This improves the detail optimization level and boundary smoothing efficiency of the completed areas, greatly enhancing the visual consistency of panoramic images and further improving the generalization ability of panoramic camera equipment in different time periods and environments. Attached Figure Description

[0044] Figure 1 A flowchart illustrating an image reflection elimination method based on panoramic photography provided in an embodiment of the present invention;

[0045] Figure 2 This is a first sub-flowchart of the image reflection elimination method based on panoramic photography provided in an embodiment of the present invention;

[0046] Figure 3 This is a second sub-flowchart of the image reflection elimination method based on panoramic photography provided in an embodiment of the present invention;

[0047] Figure 4 This is a third sub-flowchart of the image reflection elimination method based on panoramic photography provided in an embodiment of the present invention;

[0048] Figure 5 This is a block diagram of the image reflection cancellation system based on panoramic photography provided in an embodiment of the present invention;

[0049] Figure 6 A block diagram of the recording module in an image reflection cancellation system based on panoramic photography provided in an embodiment of the present invention;

[0050] Figure 7 A block diagram illustrating the components of the alignment module in an image reflection elimination system based on panoramic photography, provided in an embodiment of the present invention.

[0051] Figure 8 This is a block diagram of the update module in the panoramic camera-based image reflection elimination system provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0053] In Example 1, Figure 1 The implementation flow of the image reflection elimination method based on panoramic photography provided by the embodiment of the present invention is shown below:

[0054] S100: Using a panoramic camera, it captures panoramic images of the area under use. The infrared emitter, which is pre-installed in the area under use, illuminates the area under use according to the pre-edited traversal rules, and records the spatial projection trajectory of the infrared light spot emitted by the infrared emitter within the area under use.

[0055] The area requiring panoramic image monitoring and the corresponding panoramic camera equipment are determined. The area to be monitored is the usage area, which can be a public activity area, road surface, or other monitoring area. The panoramic camera equipment continuously collects panoramic images of the usage area. In this embodiment, the panoramic image refers to the set of image data obtained by the panoramic camera equipment capturing the entire field of view of the usage area at specific time intervals (e.g., every 1 second). A set of infrared emitters is installed in the usage area. These emitters emit infrared light signals of specific wavelengths and power. When an image reflection area is detected in the panoramic image, the usage area is systematically scanned using a traversal rule. The image reflection area refers to an abnormally bright or overexposed area in the panoramic image caused by specular reflection of ambient light by highly reflective materials (such as glass, metal, or smooth liquids). The traversal rule is the specific method of illuminating the usage area with infrared light; for example, a serpentine scanning path is used to perform line-by-line coverage illumination of the usage area. The advantage of this method is that by using infrared light to cover and illuminate the area, it is easier for panoramic camera equipment and image sensing equipment to align the images, providing a data foundation for subsequent image compensation.

[0056] The spatial projection trajectory of the infrared light spot within the usage area is recorded. The infrared light spot is the location of the highlighted response generated by the infrared emitter illuminating the usage area. The positional changes of the infrared light spot in each frame are tracked using timestamp information, recording the movement trajectory of the infrared light spot within the usage area, i.e., its spatial projection trajectory.

[0057] S200: Using an image sensing device installed in the area of ​​use, collect the movement data of infrared light spots, embed timestamps into the panoramic camera and the image sensing device, and align the spatial projection trajectory and movement data.

[0058] Identify the image sensing devices installed in the usage area. These devices, such as surveillance cameras or other ordinary cameras installed in the usage area, are capable of optical imaging of the area. Using these image sensing devices, acquire motion data of infrared light spots within the usage area, obtaining motion information of the infrared light spots at different devices and time points. Embed timestamps in both the panoramic camera and the image sensing devices to establish a cross-device time reference system, enabling precise mapping of data from different sources on the same timeline. This allows for temporal matching and alignment between the spatial projection trajectory of the infrared light spots acquired by the panoramic camera and the motion data of the infrared light spots acquired by the image sensing devices.

[0059] S300: Identify the image reflection area in the panoramic image, delineate the boundary line, and define it as the moving path of the infrared light point. Send the moving path to the infrared transmitter. Based on the image sensing device, collect real-time video data in the area to be used, extract the part composed of the moving path, define it as the reference area, generate a semantic description, perform masking processing on the image reflection area, input the masked panoramic image and semantic description into the pre-trained image restoration model, output the complete image, and send the complete image to the preset terminal to update the panoramic image.

[0060] This method detects and segments abnormally bright areas in panoramic images caused by highly reflective materials. By combining brightness threshold analysis, texture consistency analysis, and edge gradient change detection, image reflection regions are identified. These regions are areas where image reflection exists. Contour extraction is performed on these reflection regions to generate boundary lines characterizing their spatial extent. Connecting all boundary lines forms a complete spatial trajectory, which is then mapped to a guide path for infrared light points. This guide path is sent to the infrared transmitter as control commands, enabling the transmitter to perform directional scanning illumination along this path.

[0061] Using image sensing equipment, real-time video data of the area is acquired, and the movement path of infrared light spots in the real-time video data is extracted. Through target detection algorithms, the area encompassed by this movement path is located, resulting in a reference region. In other words, the reference region is the area enclosed by the movement trajectory of the infrared light spots in the real-time video data. Feature extraction and structural analysis are performed on the reference region. Combining the spatial location, boundary morphology, texture distribution, and surrounding environment of the reference region, a corresponding semantic description is generated. For example, a semantic description might be: "Water stains remaining on the ground next to a water dispenser." The semantic description is mapped to the identified image reflection areas in the panoramic image. Based on the mapping result, masking processing is performed on the image reflection areas, i.e., deleting the corresponding parts of the image reflection areas. Based on deep learning algorithms, an image inpainting model is constructed. This model can output corresponding supplementary image data based on the semantic description and the semantic features of the adjacent areas of the masked portion. The panoramic image and semantic description are input into the image restoration model, and the output is a complete image. The complete image is then sent to a preset terminal, which is a panoramic image processing terminal. At preset time intervals, snapshots are taken from the video data of the panoramic camera device, processed, and the corresponding complete image is generated. The complete image is then written back and overwrites the corresponding frame in the video data.

[0062] In Example 2, Figure 2 The first sub-flowchart of the image reflection cancellation method based on panoramic photography provided by an embodiment of the present invention is shown. The following details the steps of using a panoramic camera device to acquire panoramic images of the area to be used, and then transmitting the images via an infrared transmitter pre-installed in the area to the area to be used, according to the pre-edited traversal rules:

[0063] S101: Collect environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution.

[0064] Collect environmental data in the usage area, including: building structure, usage scenario (indoor or outdoor), and facility distribution (equipment location, structural boundaries, and obstacle distribution).

[0065] S102: Based on the environmental data, select several marker points within the usage area and set the installation location of each marker point.

[0066] Several representative marker points are selected within the usage area. These marker points primarily serve to provide visual anchors and improve the accuracy of locating image reflection areas. Auxiliary devices are then placed within these marker points to indicate their positions; for example, fragrance diffusers or indicator lights can be installed at each marker point.

[0067] In Embodiment 3, unlike Embodiment 1, the method further includes:

[0068] The panoramic image is divided into several sub-regions, and the probability level of the occurrence of image reflection areas in each sub-region is determined;

[0069] The factors influencing the probability level of occurrence are set, including at least: light, time of day, and spatial structure, and emergency response rules are created corresponding to each factor.

[0070] Based on the building structure and facility distribution within the area of ​​use, the panoramic image is divided into several sub-regions. For example, if the panoramic image is a panoramic monitoring view of the interior of an industrial plant, it can be divided into equipment operation areas, personnel passage areas, material storage areas, and wall and glass curtain wall areas, etc., according to spatial function and structural characteristics. The probability of image reflection areas appearing in each sub-region is set, i.e., the probability level, which includes high, medium, and low. Factors affecting the probability of image reflection areas appearing are identified, i.e., influencing factors. Influencing factors include lighting, time of day, and spatial structure, etc. A set of corresponding emergency response rules is set for each influencing factor. The emergency response rules are the methods for handling image reflections; for example, an emergency response rule could be manual obstruction or adjustment of the panoramic camera's position.

[0071] In Example 4, Figure 3 The second sub-flowchart of the image reflection cancellation method based on panoramic photography provided in this embodiment of the invention is shown. The following is a detailed description of the step of collecting the movement data of infrared light spots using an image sensing device installed in the area of ​​use:

[0072] S201: Build a management platform for image sensing equipment, open up registration permissions for the management platform, and identify registered personnel.

[0073] A management platform for image sensing devices is constructed, which provides unified access and data management for image sensing devices, opens up registration permissions for the management platform, and defines users who have passed identity verification as registered personnel.

[0074] S202: Receive image data uploaded by registered users, obtain sample images, and establish the correspondence between sample images and image reflection areas.

[0075] The system receives image data of the usage area taken by registered personnel and defines it as sample images, establishing a correspondence between the image reflection areas and the sample images. The advantage of this method is that it enables orderly management of sample images even when multiple image reflection areas exist within the usage area.

[0076] In Example 5, Figure 4The diagram illustrates the third sub-process flowchart of the image reflection elimination method based on panoramic camera provided in this embodiment of the invention. The following details the steps of extracting the portion composed of the moving path, defining it as a reference region, and generating a semantic description:

[0077] S301: Create a semantic parsing model, randomly select several sub-regions and define them as sample regions, manually annotate the sample regions, generate a training set, and train the semantic parsing model.

[0078] A semantic parsing model is constructed using deep learning algorithms, enabling pixel-level semantic understanding of input image data. Several sub-regions are randomly selected as sample regions, and the types of objects and reflections in the images are manually labeled. The sample regions and manually labeled content are integrated to generate a training set, which is then used to train the semantic parsing model.

[0079] S302: Input the reference region into the trained semantic parsing model and output the semantic description.

[0080] The image data corresponding to the reference region is input into the semantic parsing model, and the semantic description is output.

[0081] In Example 6, Figure 4 The diagram illustrates the third sub-process flowchart of the image reflection elimination method based on panoramic photography provided in this embodiment of the invention. The steps of inputting the masked panoramic image and semantic description into a pre-trained image inpainting model, outputting a complete image, sending the complete image to a preset terminal, and updating the panoramic image are described in detail below:

[0082] S303: According to a preset time step, extract several video frames from the image data captured by the panoramic camera device and define them as a panoramic image.

[0083] According to a preset time step, a number of video frames are extracted from the image data captured by the panoramic camera device, and the extracted video frames are defined as panoramic images. The time step can be 1 second or 1 minute, etc.

[0084] S304: Define the panoramic image with image reflection areas as a defective image, replace the defective image with the complete image, and correct the panoramic image.

[0085] Using a semantic parsing model, semantic recognition is performed on panoramic images to identify the parts of the panoramic image containing image reflections. These parts are defined as defective images. After reflection elimination processing is performed on the defective images, the obtained complete images are used to cover the defective images. The panoramic images captured by the panoramic camera are then corrected to generate image data after reflection elimination.

[0086] Figure 5 This diagram illustrates the structural block diagram of an image reflection reduction system based on panoramic photography provided in an embodiment of the present invention. The image reflection reduction system 1 based on panoramic photography includes:

[0087] The recording module 11 is used to acquire panoramic images of the area to be used using a panoramic camera device, and illuminate the area to be used by an infrared transmitter pre-installed in the area to be used according to a pre-edited traversal rule, and record the spatial projection trajectory of the infrared light spot emitted by the infrared transmitter in the area to be used.

[0088] Alignment module 12 is used to collect the movement data of infrared light spots using the image sensing device installed in the use area, embed timestamps into the panoramic camera device and the image sensing device, and align the spatial projection trajectory and movement data.

[0089] Update module 13 is used to identify image reflection areas in the panoramic image, delineate boundary lines, and define them as the movement path of infrared light points. The movement path is sent to the infrared transmitter. Based on the image sensing device, real-time video data in the area is collected, the part composed of the movement path is extracted and defined as the reference area, a semantic description is generated, the image reflection areas are masked, the masked panoramic image and semantic description are input into a pre-trained image restoration model, a complete image is output, and the complete image is sent to a preset terminal to update the panoramic image.

[0090] The segmentation module 14 is used to segment the panoramic image into several sub-regions, determine the probability level of the occurrence of image reflection areas in each sub-region, set the influencing factors of the occurrence probability level, wherein the influencing factors include at least: light, time period and spatial structure, and create emergency response rules corresponding to each influencing factor.

[0091] Figure 6 This diagram illustrates the composition of a recording module 11 in an image reflection cancellation system based on panoramic photography provided in an embodiment of the present invention. The recording module 11 includes:

[0092] The acquisition unit 111 is used to acquire environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution;

[0093] Setting unit 112 is used to select a number of marker points within the usage area based on the environmental data, and set the installation position of each marker point.

[0094] Figure 7 This diagram illustrates the composition of the alignment module 12 in the panoramic camera-based image reflection elimination system provided in an embodiment of the present invention. The alignment module 12 includes:

[0095] Construction unit 121 is used to build a management platform for image sensing equipment, open the registration permissions of the management platform, and identify registered personnel;

[0096] Unit 122 is used to receive image data uploaded by registered users, obtain sample images, and establish the correspondence between sample images and image reflection areas.

[0097] Figure 8 This diagram illustrates the structural composition of the update module 13 in the panoramic camera-based image reflection reduction system provided in an embodiment of the present invention. The update module 13 includes:

[0098] Create unit 131, which is used to create a semantic parsing model, randomly select several sub-regions and define them as sample regions, manually annotate the sample regions, generate a training set, and train the semantic parsing model;

[0099] Input unit 132 is used to input the reference region into the trained semantic parsing model and output a semantic description.

[0100] The cropping unit 133 is used to crop a number of video frames from the image data captured by the panoramic camera device according to a preset time step and define them as a panoramic image.

[0101] The correction unit 134 is used to define the panoramic image with an image reflection area as a defective image, replace the defective image with the complete image, and correct the image data.

[0102] The recording module 11 is mainly used to complete step S100, the alignment module 12 is mainly used to complete step S200, and the update module 13 is mainly used to complete step S300.

[0103] The acquisition unit 111 is mainly used to complete step S101, and the setting unit 112 is mainly used to complete step S102.

[0104] The building unit 121 is mainly used to complete step S201, and the obtaining unit 122 is mainly used to complete step S202;

[0105] The creation unit 131 is mainly used to complete step S301, the input unit 132 is mainly used to complete step S302, the interception unit 133 is mainly used to complete step S303, and the correction unit 134 is mainly used to complete step S304.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for eliminating image reflections based on panoramic photography, characterized in that, The method includes: Using panoramic camera equipment, panoramic images of the area to be used are captured. The area to be used is then illuminated by an infrared transmitter pre-installed in the area according to a pre-edited traversal rule. The spatial projection trajectory of the infrared light spot emitted by the infrared transmitter within the area to be used is recorded. Using image sensing devices installed in the usage area, the movement data of infrared light spots are collected, and timestamps are embedded into the panoramic camera and image sensing devices to align the spatial projection trajectory and movement data. The image reflection areas in the panoramic image are identified, boundary lines are delineated, and defined as the movement path of the infrared light points. The movement path is sent to the infrared transmitter. Based on the image sensing device, real-time video data in the area is collected, the part composed of the movement path is extracted and defined as the reference area, and a semantic description is generated. The image reflection areas are masked, and the masked panoramic image and semantic description are input into a pre-trained image restoration model to output a complete image. The complete image is then sent to a preset terminal to update the panoramic image.

2. The image reflection elimination method based on panoramic photography according to claim 1, characterized in that, The steps of using a panoramic camera to acquire panoramic images of the area under use, and then transmitting them via an infrared transmitter pre-installed in the area under pre-edited traversal rules, include: Collect environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution; Based on the environmental data, several marker points are selected within the usage area, and the installation location of each marker point is set.

3. The image reflection elimination method based on panoramic photography according to claim 1, characterized in that, The method further includes: The panoramic image is divided into several sub-regions, and the probability level of the occurrence of image reflection areas in each sub-region is determined; The factors influencing the probability level of occurrence are set, including at least: light, time of day, and spatial structure, and emergency response rules are created corresponding to each factor.

4. The image reflection elimination method based on panoramic photography according to claim 1, characterized in that, The step of collecting movement data of infrared light spots using image sensing devices installed within the usage area includes: Build a management platform for image sensing devices, grant registration permissions to the management platform, and identify registered users; The system receives image data uploaded by registered users, obtains sample images, and establishes a correspondence between the sample images and the image reflection areas.

5. The image reflection elimination method based on panoramic photography according to claim 3, characterized in that, The step of extracting the portion of the movement path, defining it as a reference region, and generating a semantic description includes: Create a semantic parsing model, randomly select several sub-regions and define them as sample regions, manually annotate the sample regions, generate a training set, and train the semantic parsing model; The reference region is input into the trained semantic parsing model, and the output is a semantic description.

6. The image reflection elimination method based on panoramic photography according to claim 5, characterized in that, The steps of inputting the masked panoramic image and semantic description into a pre-trained image inpainting model, outputting a complete image, and sending the complete image to a preset terminal to update the panoramic image include: According to a preset time step, a number of video frames are extracted from the image data captured by the panoramic camera device and defined as a panoramic image; A panoramic image containing image reflection areas is defined as a defective image, and the complete image is used to replace the defective image to correct the panoramic image.

7. An image reflection cancellation system based on panoramic photography, characterized in that, The system includes: The recording module is used to acquire panoramic images of the area under use using a panoramic camera device, and illuminate the area under use according to the pre-edited traversal rules via an infrared emitter pre-installed in the area under use, and record the spatial projection trajectory of the infrared light spot emitted by the infrared emitter within the area under use. The alignment module is used to collect the movement data of infrared light spots using the image sensing device installed in the area of ​​use, embed timestamps into the panoramic camera and the image sensing device, and align the spatial projection trajectory and movement data. The update module is used to identify image reflection areas in the panoramic image, delineate boundary lines, and define them as the movement path of infrared light points. The movement path is sent to the infrared transmitter. Based on the image sensing device, real-time video data in the area is collected, the part composed of the movement path is extracted and defined as the reference area, a semantic description is generated, the image reflection areas are masked, the masked panoramic image and semantic description are input into a pre-trained image restoration model, a complete image is output, and the complete image is sent to a preset terminal to update the panoramic image. The segmentation module is used to segment the panoramic image into several sub-regions, determine the probability level of the occurrence of image reflection areas in each sub-region, set the influencing factors of the occurrence probability level, wherein the influencing factors include at least: light, time of day and spatial structure, and create emergency response rules corresponding to each influencing factor.

8. The image reflection cancellation system based on panoramic photography according to claim 7, characterized in that, The recording module includes: A data acquisition unit is used to collect environmental data in the area of ​​use, wherein the environmental data includes at least: usage scenarios and facility distribution; The setting unit is used to select several marker points within the usage area based on the environmental data, and set the installation location of each marker point.

9. The image reflection cancellation system based on panoramic photography according to claim 7, characterized in that, The alignment module includes: The building unit is used to build a management platform for image sensing devices, open up registration permissions for the management platform, and identify registered users; The receiving unit is used to receive image data uploaded by registered users, obtain sample images, and establish the correspondence between sample images and image reflection areas.

10. The image reflection cancellation system based on panoramic photography according to claim 7, characterized in that, The update module includes: A creation unit is used to create a semantic parsing model. Several sub-regions are randomly selected and defined as sample regions. The sample regions are manually labeled to generate a training set for training the semantic parsing model. The input unit is used to input the reference region into the trained semantic parsing model and output a semantic description. The cropping unit is used to extract a number of video frames from the image data captured by the panoramic camera device according to a preset time step, and define them as a panoramic image; The correction unit is used to define a panoramic image with an image reflection area as a defective image, replace the defective image with the complete image, and correct the panoramic image.

Images