Image similarity determination method and device, electronic equipment and storage medium
Through the image similarity determination method, for image fusion processing in under-exposed or over-exposed scenes, the problem of low accuracy of image similarity calculation in these scenes is solved, and more efficient image feature capture and similarity analysis are achieved. .
Patent Information
- Application Number
- CN202411283845.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2025-01-21
Smart Images
Figure CN119339110A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of similarity evaluation, and in particular to a method, device, electronic device and storage medium for determining image similarity. Background Art
[0002] Image similarity determination, also known as image correlation similarity analysis, refers to the process of evaluating the degree of correlation between two or more images in terms of content, structure, features, etc. in the field of image processing and computer vision. It has wide applications in image retrieval, image classification, image registration, target recognition and other fields.
[0003] The existing image similarity determination method calculates the mean square error and signal-to-noise ratio through pixel-by-pixel differences, and determines the similarity of different images based on the mean square error calculation results and the signal-to-noise ratio calculation results. However, this image similarity determination method is affected by environmental factors such as lighting changes, perspective changes, and occlusions, and is prone to losing image details in underexposed or overexposed scenes, resulting in low accuracy in the image similarity calculation results. Summary of the invention
[0004] Based on this, it is necessary to provide a method, device, electronic device and storage medium for determining image similarity in order to solve the above technical problems.
[0005] The present application provides a method for determining image similarity, including:
[0006] Acquire multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments;
[0007] In the case where the target scene is an underexposed scene or an overexposed scene, the underexposed scene image, the normal scene image and the overexposed scene image at each acquisition moment included in the multiple frames of scene images are fused to obtain a target fused image at each acquisition moment;
[0008] Image similarity analysis is performed on the target fusion images at different acquisition moments to determine the image similarity of multiple frame scene images under the target scene.
[0009] In one embodiment, the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the multiple frames of scene images are fused to obtain a target fused image at each acquisition moment, including:
[0010] When the target scene is an underexposed scene, at each acquisition moment, the multiple frames of underexposed scene images are registered based on feature points in the multiple frames of underexposed scene images contained in the multiple frames of scene images to obtain the registered multiple frames of underexposed scene images;
[0011] The registered multiple frames of under-exposed scene images, normal scene images and over-exposed scene images are fused to obtain the target fused image at each acquisition moment.
[0012] In one embodiment, based on feature points in multiple frames of underexposed scene images contained in the multiple frames of scene images, the multiple frames of underexposed scene images are registered to obtain the registered multiple frames of underexposed scene images, including:
[0013] Extracting grayscale underexposed images, color underexposed images and saturation underexposed images from multiple frames of underexposed images respectively, to obtain multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images;
[0014] Based on the feature points in the multiple frames of grayscale underexposed images, the multiple frames of grayscale underexposed images are registered to obtain the registered multiple frames of grayscale underexposed images;
[0015] Performing weighted averaging calculation on the registered multiple frames of grayscale underexposed images to obtain a superimposed image of the multiple frames of grayscale underexposed images;
[0016] Channel superposition is performed on the superimposed images of multiple frames of grayscale underexposed images, multiple frames of color underexposed images, and multiple frames of saturation underexposed images to obtain the registered multiple frames of underexposed scene images.
[0017] In one embodiment, the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images are fused to obtain a target fused image at each acquisition moment, including:
[0018] According to the weighted values corresponding to the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images, the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images are weighted summed to obtain the target fused image at each acquisition moment.
[0019] In one embodiment, the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the multiple frames of scene images are fused to obtain a target fused image at each acquisition moment, including:
[0020] In the case where the target scene is an overexposed scene, at each acquisition moment, fusion processing is performed based on the underexposed scene image, the normal scene image, and the overexposed scene image to obtain the target fused image at each acquisition moment.
[0021] In one embodiment, it further includes:
[0022] Get the preview image of the current scene;
[0023] Convert the grayscale component data contained in the preview image into a grayscale component histogram;
[0024] Determine the scene parameters of the current scene based on the grayscale values in the grayscale component histogram and the number of pixels corresponding to the grayscale values;
[0025] If the scene parameters of the current scene are the scene parameters of the target scene, the current scene is determined to be the target scene.
[0026] In one embodiment, it further includes:
[0027] In the target scene, a similarity image is generated according to the image similarity of multiple frames of scene images and the pixel positions of the multiple frames of scene images;
[0028] Convert the similarity image into a similarity grayscale image;
[0029] Convert the similarity grayscale image into a similarity binary image;
[0030] Based on the binary data of each point in the similarity binary map, similar areas and non-similar areas are determined from the similarity image.
[0031] The present application provides an image similarity determination device, including:
[0032] An image acquisition module is used to acquire multiple frames of scene images under a target scene, wherein the multiple frames of scene images include scene images at different acquisition moments;
[0033] An image fusion module is used to fuse the underexposed scene image, the normal scene image and the overexposed scene image at each acquisition moment contained in the multiple frames of scene images when the target scene is an underexposed scene or an overexposed scene, so as to obtain a target fused image at each acquisition moment;
[0034] The image similarity determination module is used to perform image similarity analysis on target fusion images at different acquisition moments to determine the image similarity of multiple frames of scene images under the target scene.
[0035] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the image similarity determination and analysis method provided in any embodiment of the present application are implemented.
[0036] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for determining image similarity provided in any embodiment of the present application are implemented.
[0037] The image similarity determination method, device, electronic device and storage medium provided in the embodiments of the present application obtain multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, the underexposed scene images, normal scene images and overexposed scene images at each acquisition moment contained in the multiple frames of scene images are fused to obtain the target fused image at each acquisition moment; image similarity analysis is performed on the target fused images at different acquisition moments to determine the image similarity of the multiple frames of scene images under the target scene. In this way, in the underexposed scene or the overexposed scene, the detail information of the different scene images is retained by fusion processing the underexposed scene images, normal scene images and overexposed scene images at the acquisition moment, effectively suppressing the influence of environmental factors such as illumination changes, perspective changes and occlusion on the detail information of the different scene images, achieving comprehensive capture of image features and improving the accuracy of the image similarity calculation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a schematic flow chart of a method for determining image similarity in one embodiment;
[0039] Figure 2 is a schematic structural diagram of an image similarity determination device in one embodiment;
[0040] Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device in an embodiment. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0042] In the related art, a comparison algorithm of key points such as feature texture is also used to calculate the image texture structure to determine the image similarity, or a color histogram is directly used to compare the color distribution of the image to determine the image similarity, or a structural similarity method combining brightness, color, and position comparison is used to determine the image similarity. However, any of the above image similarity determination methods is also affected by environmental factors such as lighting changes, viewing angle changes, and occlusion, and is prone to losing image details in underexposed or overexposed scenes, resulting in low accuracy of image similarity calculation results.
[0043] In one embodiment, Figure 1As shown, a method for determining image similarity is provided. This embodiment uses the method applied to an electronic device as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including an electronic device and a server, and is implemented through the interaction between the electronic device and the server.
[0044] In this embodiment, the method includes the following steps:
[0045] Step 102: Acquire multiple frames of scene images in the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments.
[0046] The target scene refers to the scene that the electronic device is in. Optionally, the target scene is any one of an underexposed scene, an overexposed scene, and a normal scene.
[0047] The multiple frames of scene images refer to real scene images collected in the target scene. Specifically, the multiple frames of scene images are collected at different collection times to obtain scene images at different collection times.
[0048] Specifically, in any target scene with illumination and occlusion, the user activates the shutter button of the camera of the electronic device, so that the camera shoots the scene at different acquisition times under any viewing angle of the target scene to obtain multiple frames of scene images.
[0049] Step 104: When the target scene is an underexposed scene or an overexposed scene, the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the multiple frames of scene images are fused to obtain a target fused image at each acquisition moment.
[0050] Among them, underexposed scenes can be understood as dark environment scenes, that is, scenes with low brightness, and overexposed scenes can be understood as bright environment scenes, that is, scenes with high brightness.
[0051] Among them, underexposed scene images refer to images with lower brightness collected in underexposed scenes or overexposed scenes, normal scene images refer to images with moderate brightness collected in underexposed scenes or overexposed scenes, and overexposed scene images refer to images with higher brightness collected in underexposed scenes or overexposed scenes.
[0052] The target fused image refers to a scene image that fuses the image features of an underexposed scene image, a normal scene image, and an overexposed scene image at each acquisition moment.
[0053] It is understandable that underexposed scenes test the quality of cameras on electronic devices, and images are easily mixed with noise. Therefore, it is necessary to collect underexposed scene images, normal scene images, and overexposed scene images in underexposed scenes, and fuse the collected scene images to retain more detailed information of the scene images, and at the same time, it can suppress the influence of environmental factors such as lighting changes, perspective changes, and occlusions on the detailed information of different scene images. Similarly, the highlight areas of overexposed scenes do have obvious lack of details and brightness in dark areas. Therefore, it is necessary to collect underexposed scene images, normal scene images, and overexposed scene images in overexposed scenes, and fuse the collected scene images to retain more detailed information of the scene images, and at the same time, it can suppress the influence of environmental factors such as lighting changes, perspective changes, and occlusions on the detailed information of different scene images.
[0054] Step 106: Perform image similarity analysis on the target fusion images at different acquisition moments to determine the image similarity of multiple frames of scene images in the target scene.
[0055] Specifically, in a target scene, the electronic device may input target fused images at different acquisition moments into a preset algorithm to determine image similarities of multiple frames of scene images in the target scene.
[0056] Optionally, the preset algorithm includes but is not limited to a structural similarity algorithm, a histogram algorithm, etc., and may also be other image similarity analysis algorithms.
[0057] In the above-mentioned method for determining image similarity, a plurality of scene images of a target scene are obtained, wherein the plurality of scene images include scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, the underexposed scene images, normal scene images, and overexposed scene images at each acquisition moment contained in the plurality of scene images are fused to obtain a target fused image at each acquisition moment; image similarity analysis is performed on the target fused images at different acquisition moments to determine the image similarity of the plurality of scene images of the target scene. In this way, in an underexposed scene or an overexposed scene, by fusion processing the underexposed scene images, normal scene images, and overexposed scene images at the acquisition moment, the detail information of the different scene images is retained, and the influence of environmental factors such as illumination change, perspective change, and occlusion on the detail information of the different scene images is effectively suppressed, and the image features are fully captured, thereby improving the accuracy of the image similarity calculation results.
[0058] In one embodiment, a fusion process is performed on the underexposed scene images, normal scene images, and overexposed scene images at each acquisition moment included in the multi-frame scene images to obtain a target fused image at each acquisition moment, including: when the target scene is an underexposed scene, at each acquisition moment, the multi-frame underexposed scene images are registered based on feature points in the multi-frame underexposed scene images included in the multi-frame scene images to obtain the registered multi-frame underexposed scene images; and the registered multi-frame underexposed scene images, normal scene images, and overexposed scene images are fused to obtain the target fused image at each acquisition moment.
[0059] In this embodiment, in an underexposed scene, the number of underexposed scene images at each acquisition moment is at least two frames, and the number of normal scene images and overexposed scene images are both one frame. For example, in an underexposed scene, five frames of scene images are acquired at each acquisition moment, specifically including three frames of underexposed scene images, one frame of normal scene image, and one frame of overexposed scene image.
[0060] Among them, based on the feature points in the multi-frame under-exposed scene images contained in the multi-frame scene images, the multi-frame under-exposed scene images are aligned to obtain the aligned multi-frame under-exposed scene images, including: respectively extracting grayscale under-exposed images, color under-exposed images and saturation under-exposed images from the multi-frame under-exposed images to obtain multi-frame grayscale under-exposed images, multi-frame color under-exposed images and multi-frame saturation under-exposed images; based on the feature points in the multi-frame grayscale under-exposed images, the multi-frame grayscale under-exposed images are aligned to obtain the aligned multi-frame grayscale under-exposed images; weighted averaging calculation is performed on the aligned multi-frame grayscale under-exposed images to obtain the superimposed image of the multi-frame grayscale under-exposed images; channel superposition is performed on the superimposed image of the multi-frame grayscale under-exposed images, the multi-frame color under-exposed images and the multi-frame saturation under-exposed images to obtain the aligned multi-frame under-exposed scene images.
[0061] Specifically, when the multiple frames of underexposed images are color-coded images (YUV images), or after the multiple frames of underexposed images are converted from a color space (RGB space) to a color-coded space (YUV space), first, the electronic device extracts a grayscale underexposed image (Y channel image), a color underexposed image (U channel image) and a saturation underexposed image (V channel image) from the multiple frames of underexposed images respectively; then, the electronic device aligns the multiple frames of grayscale underexposed images based on feature points in the multiple frames of grayscale underexposed images so as to align the multiple frames of grayscale underexposed images and obtain the registered multiple frames of grayscale underexposed images; then, the electronic device performs weighted averaging calculation based on the weights corresponding to the registered multiple frames of grayscale underexposed images and obtains a superimposed image of the multiple frames of grayscale underexposed images; then, the electronic device continues to perform channel superposition of the superimposed image of the multiple frames of grayscale underexposed images, the multiple frames of color underexposed images and the multiple frames of saturation underexposed images according to channels to obtain the registered multiple frames of underexposed scene images, thereby improving the signal-to-noise ratio of the multiple frames of underexposed scene images.
[0062] Among them, the registered multiple frames of under-exposed scene images, normal scene images and over-exposed scene images are fused to obtain the target fused image at each acquisition moment, including: according to the weighted values corresponding to the registered multiple frames of under-exposed scene images, normal scene images and over-exposed scene images, the registered multiple frames of under-exposed scene images, normal scene images and over-exposed scene images are weighted summed to obtain the target fused image at each acquisition moment.
[0063] Specifically, the electronic device obtains pre-configured weighted values for the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images, and then performs weighted summation according to the weighted values corresponding to the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images, so that the multiple scene images are superimposed to obtain a target fused image at each acquisition moment in the underexposed scene, thereby improving the detail information of the image.
[0064] Optionally, the weighted value corresponding to the normal scene image can be higher than the weighted values corresponding to the multiple frames of underexposed scene images and overexposed scene images after alignment, so that the target fused image at each acquisition moment can integrate more image information of the normal scene image to improve the ratio of image detail information to noise.
[0065] In the above scheme, when the target scene is an underexposed scene, by aligning multiple frames of underexposed images and fusing the underexposed scene images, normal scene images and overexposed scene images, the ratio of image detail information to noise can be improved, thereby improving the detail information of the image in the underexposed scene.
[0066] In one embodiment, when the target scene is an overexposed scene, at each acquisition moment, a fusion process is performed based on the underexposed scene image, the normal scene image, and the overexposed scene image to obtain a target fused image at each acquisition moment.
[0067] Specifically, when the target scene is an overexposed scene, the electronic device only collects one frame of underexposed scene image, one frame of normal scene image and one frame of overexposed scene image, and then directly obtains pre-configured weighted values of the one frame of underexposed scene image, one frame of normal scene image and one frame of overexposed scene image, and performs weighted summation on the one frame of underexposed scene image, one frame of normal scene image and one frame of overexposed scene image, so that multiple scene images are superimposed to obtain a target fused image at each collection moment in the overexposed scene, thereby improving the detail information of the image.
[0068] Optionally, the weighted value corresponding to the normal scene image can be higher than the weighted values corresponding to the underexposed scene image and the overexposed scene image after registration, so that the target fused image at each acquisition moment can fuse more image information of the normal scene image to improve the ratio of image detail information to noise.
[0069] In the above scheme, when the target scene is an overexposed scene, the three scene images can be fused based on the weighted values corresponding to the underexposed scene image, the normal scene image and the overexposed scene image, thereby improving the ratio of image detail information to noise, thereby improving the image detail information.
[0070] In one embodiment, when the target scene is a normal scene, image similarity analysis is performed on two frames of normal scene images at different acquisition times included in the multiple frames of scene images to obtain image similarity of the multiple frames of scene images under the target scene.
[0071] Specifically, when the target scene is a normal scene, the electronic device directly obtains two frames of normal scene images at different acquisition times, and inputs the two frames of normal scene images at different acquisition times into a preset algorithm for image similarity analysis to obtain image similarity of multiple frames of scene images under the target scene.
[0072] In the above scheme, when the target scene is a normal scene, image similarity analysis can be performed directly based on two frames of normal scene images acquired at different times, thereby simplifying the process of image similarity analysis.
[0073] In one embodiment, the method also includes: obtaining a preview image of the current scene; converting the format of the grayscale component data contained in the preview image to obtain a grayscale component histogram; determining the scene parameters of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value; if the scene parameters of the current scene are the scene parameters of the target scene, determining that the current scene is the target scene.
[0074] The current image refers to the scene where the electronic device is located before the target scene is determined.
[0075] The preview image of the current scene refers to an image displayed in an image preview frame of the electronic device when a shutter button of a camera on the electronic device is not activated.
[0076] The preview image is a YUV image or an RGB image. Specifically, when the preview image is a YUV image, the grayscale component data is directly extracted from the preview image, or when the preview image is an RGB image, the preview image is first converted from the RGB space to the YUV space, and then the grayscale component data is extracted from the preview image.
[0077] The grayscale component histogram refers to the binary image corresponding to the grayscale (Y) component in the preview image. Each grayscale level in the grayscale component histogram uniquely corresponds to the number of pixels.
[0078] The scene parameters of the current scene include brightness and contrast.
[0079] Specifically, after the electronic device obtains a preview image of the current scene, it extracts grayscale component data from the preview image and converts the grayscale component data into a grayscale component histogram, and then determines the brightness and contrast of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value. Since different scenes uniquely correspond to specific brightness and specific contrast, if the brightness and contrast of the current scene are the brightness and contrast of the target scene, the current scene is determined to be the target scene.
[0080] In the above scheme, by analyzing the gray component histogram corresponding to the gray component data of the preview image of the current scene, the scene parameters of the current scene are determined, thereby accurately determining whether the current scene is the target scene, facilitating image similarity analysis under the target scene.
[0081] In one embodiment, the method further includes: generating a similarity image according to the image similarity of multiple frames of scene images and the pixel positions of the multiple frames of scene images in the target scene; converting the similarity image into a similarity grayscale image; converting the similarity grayscale image into a similarity binary image; and determining similar areas and non-similar areas from the similarity image based on the binary data of each point in the similarity binary image.
[0082] The similarity image is an image generated by the image similarity and pixel positions of multiple frames of scene images.
[0083] Specifically, the electronic device first converts the determined similarity image into a similarity grayscale image so that the image value is scaled to the range of 0-255, then converts the similarity grayscale image into a similarity binary image so that the image value is further converted into two values of 0 and 1, and then reads the binary data of each point from the similarity binary image, and uses the area with the binary data of 0 as the non-similar area of the multi-frame scene image in the target scene, and at the same time, uses the area with the binary data of 1 as the similar area of the multi-frame scene image in the target scene.
[0084] In the above scheme, by further analyzing the image similarity of multiple scene frames and the pixel positions of multiple scene frames, similar areas and non-similar areas of multiple scene frames are determined, so as to obtain more results for user reference in the image similarity analysis process.
[0085] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0086] In one embodiment, Figure 2 As shown, a device for determining image similarity is provided, comprising:
[0087] An image acquisition module 201 is used to acquire multiple frames of scene images under a target scene, wherein the multiple frames of scene images include scene images at different acquisition moments;
[0088] The image fusion module 202 is used to fuse the underexposed scene image, the normal scene image and the overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain the target fused image at each acquisition moment when the target scene is an underexposed scene or an overexposed scene;
[0089] The image similarity determination module 203 is used to perform image similarity analysis on the target fused images at different acquisition moments to determine the image similarity of multiple frames of scene images in the target scene.
[0090] Optionally, the image fusion module 202 includes:
[0091] A registration unit, for registering the multiple frames of underexposed scene images based on feature points in the multiple frames of underexposed scene images contained in the multiple frames of scene images at each acquisition moment when the target scene is an underexposed scene, to obtain the registered multiple frames of underexposed scene images;
[0092] The fusion unit is used to fuse the registered multiple frames of under-exposed scene images, normal scene images and over-exposed scene images to obtain a target fused image at each acquisition moment.
[0093] Optionally, the registration unit comprises:
[0094] An extraction subunit is used to extract grayscale underexposed images, color underexposed images and saturation underexposed images from the multiple frames of underexposed images, so as to obtain multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images;
[0095] A registration subunit, used for registering the multiple frames of grayscale underexposed images based on feature points in the multiple frames of grayscale underexposed images to obtain the registered multiple frames of grayscale underexposed images;
[0096] A weighted averaging subunit is used to perform weighted averaging calculation on the registered multiple frames of grayscale underexposed images to obtain a superimposed image of the multiple frames of grayscale underexposed images;
[0097] The channel superposition subunit is used to perform channel superposition on the superimposed images of multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images to obtain the registered multiple frames of underexposed scene images.
[0098] Optionally, a fusion unit for:
[0099] According to the weighted values corresponding to the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images, the registered multiple frames of underexposed scene images, normal scene images and overexposed scene images are weighted summed to obtain the target fused image at each acquisition moment.
[0100] Optionally, the image fusion module 202 is used to:
[0101] In the case where the target scene is an overexposed scene, at each acquisition moment, fusion processing is performed based on the underexposed scene image, the normal scene image, and the overexposed scene image to obtain the target fused image at each acquisition moment.
[0102] Optionally, it also includes:
[0103] The preview image module is used to obtain the preview image of the current scene;
[0104] The format conversion module is used to convert the grayscale component data contained in the preview image into a format to obtain a grayscale component histogram;
[0105] A scene parameter determination module, used to determine the scene parameters of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value;
[0106] The target scene determination module is used to determine that the current scene is the target scene if the scene parameters of the current scene are the scene parameters of the target scene.
[0107] Optionally, it also includes:
[0108] A similarity image generation module is used to generate a similarity image according to the image similarity of multiple frames of scene images and the pixel positions of the multiple frames of scene images in a target scene;
[0109] A first image conversion module, used for converting the similarity image into a similarity grayscale image;
[0110] The second image conversion module is used to convert the similarity grayscale image into a similarity binary image;
[0111] The region determination module is used to determine similar regions and non-similar regions from the similarity image based on the binary data of each point in the similarity binary image.
[0112] The image similarity determination device provided in the embodiment of the present application obtains multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, the underexposed scene images, normal scene images, and overexposed scene images at each acquisition moment contained in the multiple frames of scene images are fused to obtain a target fused image at each acquisition moment; image similarity analysis is performed on the target fused images at different acquisition moments to determine the image similarity of the multiple frames of scene images under the target scene. In this way, in an underexposed scene or an overexposed scene, by fusion processing the underexposed scene images, normal scene images, and overexposed scene images at the acquisition moment, the detail information of different scene images is retained, and the influence of environmental factors such as illumination changes, perspective changes, and occlusion on the detail information of different scene images is effectively suppressed, and comprehensive capture of image features is achieved, thereby improving the accuracy of image similarity calculation results.
[0113] For the specific definition of the image similarity determination device, please refer to the definition of the image similarity determination method above, which will not be repeated here. Each module in the above-mentioned image similarity determination device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0114] In one embodiment, an electronic device is provided. The electronic device may be a terminal device or a server, and its internal structure diagram may be as follows: Figure 3 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for classifying short messages is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0115] Those skilled in the art will understand that Figure 3The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0116] In one embodiment, the image similarity determination device provided by the present application can be implemented in the form of a computer program. The computer program can be Figure 3 The memory of the electronic device may store various program modules constituting the image similarity determination device, for example, Figure 2 The image acquisition module 201, the image fusion module 202, and the image similarity determination module 203 are shown. The computer program composed of various program modules enables the processor to execute the steps of the image similarity determination method of each embodiment of the present application described in this specification.
[0117] For example, Figure 3 The electronic device shown can be Figure 2 The image acquisition module 201 in the device shown is used to acquire multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; the image fusion module 202 is used to fuse the underexposed scene images, normal scene images and overexposed scene images at each acquisition moment contained in the multiple frames of scene images when the target scene is an underexposed scene or an overexposed scene, so as to obtain a target fused image at each acquisition moment; the image similarity determination module 203 is used to perform image similarity analysis on the target fused images at different acquisition moments, so as to determine the image similarity of the multiple frames of scene images under the target scene.
[0118] In one embodiment, when the processor executes the computer program, the following steps are also implemented: acquiring multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, fusing the underexposed scene images, normal scene images, and overexposed scene images at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment; performing image similarity analysis on the target fused images at different acquisition moments to determine the image similarity of the multiple frames of scene images under the target scene.
[0119] In one embodiment, the processor further implements the following steps when executing the computer program: fusing the underexposed scene images, normal scene images and overexposed scene images at each acquisition moment contained in the multi-frame scene images to obtain a target fused image at each acquisition moment, including: when the target scene is an underexposed scene, at each acquisition moment, registering the multi-frame underexposed scene images based on feature points in the multi-frame underexposed scene images contained in the multi-frame scene images to obtain registered multi-frame underexposed scene images; fusing the registered multi-frame underexposed scene images, normal scene images and overexposed scene images to obtain a target fused image at each acquisition moment.
[0120] In one embodiment, the processor further implements the following steps when executing the computer program: based on the feature points in the multiple frames of underexposed scene images contained in the multiple frames of scene images, the multiple frames of underexposed scene images are registered to obtain the registered multiple frames of underexposed scene images, including: respectively extracting grayscale underexposed images, color underexposed images and saturation underexposed images from the multiple frames of underexposed images to obtain multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images; based on the feature points in the multiple frames of grayscale underexposed images, the multiple frames of grayscale underexposed images are registered to obtain the registered multiple frames of grayscale underexposed images; performing weighted averaging calculation on the registered multiple frames of grayscale underexposed images to obtain a superimposed image of the multiple frames of grayscale underexposed images; performing channel superposition on the superimposed image of the multiple frames of grayscale underexposed images, the multiple frames of color underexposed images and the multiple frames of saturation underexposed images to obtain the registered multiple frames of underexposed scene images.
[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fusing the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images to obtain a target fused image at each acquisition moment, including: performing weighted summation on the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images according to weighted values corresponding to the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images, respectively, to obtain a target fused image at each acquisition moment.
[0122] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fusing the underexposed scene image, normal scene image and overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment, including: when the target scene is an overexposed scene, at each acquisition moment, fusing the underexposed scene image, normal scene image and overexposed scene image to obtain a target fused image at each acquisition moment.
[0123] In one embodiment, the processor implements the following steps when executing the computer program: including: obtaining a preview image of the current scene; converting the format of the grayscale component data contained in the preview image to obtain a grayscale component histogram; determining the scene parameters of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value; if the scene parameters of the current scene are the scene parameters of the target scene, determining that the current scene is the target scene.
[0124] In one embodiment, the processor implements the following steps when executing the computer program: including: in the target scene, generating a similarity image according to the image similarity of multiple frames of scene images and the pixel positions of the multiple frames of scene images; converting the similarity image into a similarity grayscale image; converting the similarity grayscale image into a similarity binary image; and determining similar areas and non-similar areas from the similarity image based on the binary data of each point in the similarity binary image.
[0125] In an embodiment of the present application, a multi-frame scene image of a target scene is obtained, wherein the multi-frame scene image includes scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment contained in the multi-frame scene image are fused to obtain a target fused image at each acquisition moment; image similarity analysis is performed on the target fused images at different acquisition moments to determine the image similarity of the multi-frame scene images of the target scene. In this way, in an underexposed scene or an overexposed scene, by fusion processing the underexposed scene image, the normal scene image, and the overexposed scene image at the acquisition moment, the detail information of the different scene images is retained, and the influence of environmental factors such as illumination change, perspective change, and occlusion on the detail information of the different scene images is effectively suppressed, and the comprehensive capture of image features is achieved, and the accuracy of the image similarity calculation results is improved.
[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: acquiring a plurality of scene image frames of a target scene, wherein the plurality of scene image frames include scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, fusing the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the plurality of scene image frames to obtain a target fused image at each acquisition moment; performing image similarity analysis on the target fused images at different acquisition moments to determine the image similarity of the plurality of scene image frames of the target scene.
[0127] In one embodiment, the processor further implements the following steps when executing the computer program: fusing the underexposed scene images, normal scene images and overexposed scene images at each acquisition moment contained in the multi-frame scene images to obtain a target fused image at each acquisition moment, including: when the target scene is an underexposed scene, at each acquisition moment, registering the multi-frame underexposed scene images based on feature points in the multi-frame underexposed scene images contained in the multi-frame scene images to obtain registered multi-frame underexposed scene images; fusing the registered multi-frame underexposed scene images, normal scene images and overexposed scene images to obtain a target fused image at each acquisition moment.
[0128] In one embodiment, the processor further implements the following steps when executing the computer program: based on the feature points in the multiple frames of underexposed scene images contained in the multiple frames of scene images, the multiple frames of underexposed scene images are registered to obtain the registered multiple frames of underexposed scene images, including: respectively extracting grayscale underexposed images, color underexposed images and saturation underexposed images from the multiple frames of underexposed images to obtain multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images; based on the feature points in the multiple frames of grayscale underexposed images, the multiple frames of grayscale underexposed images are registered to obtain the registered multiple frames of grayscale underexposed images; performing weighted averaging calculation on the registered multiple frames of grayscale underexposed images to obtain a superimposed image of the multiple frames of grayscale underexposed images; performing channel superposition on the superimposed image of the multiple frames of grayscale underexposed images, the multiple frames of color underexposed images and the multiple frames of saturation underexposed images to obtain the registered multiple frames of underexposed scene images.
[0129] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fusing the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images to obtain a target fused image at each acquisition moment, including: performing weighted summation on the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images according to weighted values corresponding to the registered multiple frames of underexposed scene images, normal scene images, and overexposed scene images, respectively, to obtain a target fused image at each acquisition moment.
[0130] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fusing the underexposed scene image, normal scene image and overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment, including: when the target scene is an overexposed scene, at each acquisition moment, fusing the underexposed scene image, normal scene image and overexposed scene image to obtain a target fused image at each acquisition moment.
[0131] In one embodiment, the processor implements the following steps when executing the computer program: including: obtaining a preview image of the current scene; converting the format of the grayscale component data contained in the preview image to obtain a grayscale component histogram; determining the scene parameters of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value; if the scene parameters of the current scene are the scene parameters of the target scene, determining that the current scene is the target scene.
[0132] In one embodiment, the processor implements the following steps when executing the computer program: including: in the target scene, generating a similarity image according to the image similarity of multiple frames of scene images and the pixel positions of the multiple frames of scene images; converting the similarity image into a similarity grayscale image; converting the similarity grayscale image into a similarity binary image; and determining similar areas and non-similar areas from the similarity image based on the binary data of each point in the similarity binary image.
[0133] In an embodiment of the present application, a multi-frame scene image of a target scene is obtained, wherein the multi-frame scene image includes scene images at different acquisition moments; when the target scene is an underexposed scene or an overexposed scene, the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment contained in the multi-frame scene image are fused to obtain a target fused image at each acquisition moment; image similarity analysis is performed on the target fused images at different acquisition moments to determine the image similarity of the multi-frame scene images of the target scene. In this way, in an underexposed scene or an overexposed scene, by fusion processing the underexposed scene image, the normal scene image, and the overexposed scene image at the acquisition moment, the detail information of the different scene images is retained, and the influence of environmental factors such as illumination change, perspective change, and occlusion on the detail information of the different scene images is effectively suppressed, and the comprehensive capture of image features is achieved, and the accuracy of the image similarity calculation results is improved.
[0134] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static random access memory (SRAM) and dynamic random access memory (DRAM).
[0135] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0136] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A method for determining image similarity, characterized in that: include: Acquire multiple frames of scene images under the target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; In the case where the target scene is an underexposed scene or an overexposed scene, fusing the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment; Image similarity analysis is performed on target fusion images at different acquisition moments to determine image similarity of multiple frames of scene images under the target scene.
2. The method according to claim 1, characterized in that The fusing process of the underexposed scene image, the normal scene image and the overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment includes: In the case where the target scene is the underexposed scene, at each acquisition moment, based on feature points in the multiple frames of underexposed scene images included in the multiple frames of scene images, the multiple frames of underexposed scene images are registered to obtain the registered multiple frames of underexposed scene images; The registered multiple frames of underexposed scene images, the normal scene images and the overexposed scene images are fused to obtain the target fused image at each acquisition moment.
3. The method according to claim 2, characterized in that The registering the multiple frames of underexposed scene images based on the feature points in the multiple frames of underexposed scene images included in the multiple frames of scene images to obtain the registered multiple frames of underexposed scene images includes: Extracting grayscale underexposed images, color underexposed images and saturation underexposed images from the multiple frames of underexposed images respectively, to obtain multiple frames of grayscale underexposed images, multiple frames of color underexposed images and multiple frames of saturation underexposed images; Based on the feature points in the multiple frames of grayscale underexposed images, registering the multiple frames of grayscale underexposed images to obtain the registered multiple frames of grayscale underexposed images; Performing weighted averaging calculation on the registered multiple frames of grayscale underexposed images to obtain a superimposed image of the multiple frames of grayscale underexposed images; Channel superposition is performed on the superimposed image of the multiple frames of grayscale underexposed images, the multiple frames of color underexposed images, and the multiple frames of saturation underexposed images to obtain the registered multiple frames of underexposed scene images.
4. The method according to claim 2, characterized in that: The fusing process of the registered multiple frames of underexposed scene images, the normal scene images and the overexposed scene images to obtain the target fused image at each acquisition moment includes: According to the weighted values respectively corresponding to the registered multiple frames of underexposed scene images, the normal scene images and the overexposed scene images, weighted summation is performed on the registered multiple frames of underexposed scene images, the normal scene images and the overexposed scene images to obtain the target fused image at each acquisition moment.
5. The method according to claim 1, characterized in that: The fusing process of the underexposed scene image, the normal scene image and the overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment includes: In the case where the target scene is the overexposed scene, at each acquisition moment, a fusion process is performed based on the underexposed scene image, the normal scene image and the overexposed scene image to obtain the target fused image at each acquisition moment.
6. The method according to claim 1, characterized in that Also includes: Get the preview image of the current scene; Converting the grayscale component data contained in the preview image into a grayscale component histogram; Determining a scene parameter of the current scene based on each grayscale value in the grayscale component histogram and the number of pixels corresponding to each grayscale value; If the scene parameters of the current scene are the scene parameters of the target scene, the current scene is determined to be the target scene.
7. The method according to claim 1, characterized in that Also includes: In the target scene, generating a similarity image according to the image similarities of the multiple frames of scene images and the pixel positions of the multiple frames of scene images; Converting the similarity image into a similarity grayscale image; Converting the similarity grayscale image into a similarity binary image; Based on the binarized data of each point in the similarity binarized image, similar regions and non-similar regions are determined from the similarity image.
8. An image similarity determination device, characterized in that: include: An image acquisition module, used to acquire multiple frames of scene images under a target scene, wherein the multiple frames of scene images include scene images at different acquisition moments; An image fusion module, configured to, when the target scene is an underexposed scene or an overexposed scene, fuse the underexposed scene image, the normal scene image, and the overexposed scene image at each acquisition moment included in the multiple frames of scene images to obtain a target fused image at each acquisition moment; The image similarity determination module is used to perform image similarity analysis on target fusion images at different acquisition moments to determine the image similarity of multiple frames of scene images under the target scene.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.