Generation area determination and spot generation method, device, medium and program product

Abstract

The present application provides a method for determining a light spot generation area, a light spot generation method, a device, a medium and a program product, which are intended to accurately determine the position of the light spot generation area in an image. The method includes: cutting the image to be detected into multiple image blocks, and recording the position of each image block in the image to be detected; extracting the image features of the multiple image blocks; inputting the image features of the multiple image blocks into a light spot generation area determination model to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area, wherein the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature of the sample image block, and the label characterizes whether the sample image block is a light spot generation area; according to the binary classification results of each of the multiple image blocks and their respective positions in the image to be detected, determine the position of the light spot generation area in the image to be detected.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for determining a light spot generation area, a light spot generation method, a device, a medium and a program product. Background Art

[0002] The flare effect of an image refers to a light and shadow effect of a blurred scene, which often gives people a romantic feeling. Figure 1 A light spot effect diagram is shown, in which the brighter circle in the blurred background is the light spot.

[0003] In the related art, the entire image is directly analyzed, identified and blurred. The resulting blurred image only contains blurred areas, and it is difficult to form a bright light spot with clear edges. In order to generate a light spot in an image, it is first necessary to determine the location of the generated light spot. How to determine the location of the generated light spot in an image is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide a method for determining a light spot generation area, a light spot generation method, a device, a medium and a program product to overcome the above problems or at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present application, a method for determining a light spot generation area is provided, comprising:

[0006] Cutting the image to be detected into multiple image blocks, and recording the position of each image block in the image to be detected;

[0007] extracting image features of the plurality of image blocks;

[0008] Inputting the image features of the multiple image blocks into a light spot generation area determination model to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area, wherein the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature of the sample image block, and the label represents whether the sample image block is a light spot generation area;

[0009] The position of the light spot generation area in the image to be detected is determined according to the binary classification results of each of the multiple image blocks and their respective positions in the image to be detected.

[0010] Optionally, the sample image blocks used for training the light spot generation area determination model are obtained according to the following steps:

[0011] Acquire a plurality of sample image pairs, each sample image pair comprising a sample image with light spots and a sample image without light spots that has the same image content as the sample image with light spots;

[0012] Selecting a light spot region of the light spot sample image in each of the sample image pairs to obtain a sample image with a light spot position frame;

[0013] The sample image with light spot having the light spot position frame and the corresponding sample image without light spot are cut with the same granularity to obtain a plurality of sample image block pairs, each of which comprises: a first sample image block derived from the sample image with the light spot position frame, and a second sample image block derived from the sample image without light spot, wherein the position of the first sample image block in the sample image with the light spot position frame is the same as the position of the second sample image block in the sample image without light spot, and the two sample image blocks in each sample image block pair are derived from two images of the same sample image pair respectively;

[0014] Determine whether the first sample image block belongs to the area where the light spot is located according to the ratio of the area belonging to the light spot position frame in each first sample image block to the total area of ​​the first sample image block;

[0015] Determine the sample image block pair to which the first sample image block belonging to the area where the light spot is located as the light spot sample image block pair;

[0016] Determine the second sample image block belonging to the spot sample image block pair as a sample image block carrying a positive label;

[0017] The second sample image block that does not belong to the spot sample image block pair is determined as a sample image block carrying a negative label.

[0018] Optionally, the step of training the light spot generation area determination model includes:

[0019] Extracting image features of each sample image block used for training the light spot generation area determination model;

[0020] Inputting the image features of each sample image block into the initial model to obtain a binary classification prediction result of whether each sample image block is a light spot generation area;

[0021] Establishing a loss function according to the difference between the binary classification prediction result of each sample image block and the label carried by the sample image block;

[0022] The model parameters of the initial model are updated based on the loss function to obtain the light spot generation area determination model.

[0023] Optionally, the light spot generation area determination model is obtained by fusing multiple determination sub-models according to their corresponding weights, and each of the determination sub-models is obtained by supervised training of an initial sub-model using image features of a sample image block and a label carried by the sample image block.

[0024] Optionally, one of the multiple determination sub-models is a color value model, which is obtained by supervised training of an initial color value model using a first predicted probability value of a sample image block and a label carried by the sample image block, and the first predicted probability value represents: the initial color value model predicts the probability value of the sample image block as a light spot generation area based on the color value characteristics of the sample image block.

[0025] Optionally, one of the multiple determination sub-models is a gray value model, which is obtained by supervised training of an initial gray value model using a second predicted probability value of the sample image block and a label carried by the sample image block, and the second predicted probability value represents: the initial gray value model predicts the probability value of the sample image block as a light spot generation area based on the gray value characteristics of the sample image block.

[0026] Optionally, one of the multiple determination sub-models is a brightness value model; the brightness value model is obtained by performing supervised training on an initial brightness value model using a third predicted probability value of the sample image block and a label carried by the sample image block, and the third predicted probability value represents: the initial brightness value model predicts the probability value of the sample image block as a light spot generation area based on the brightness value characteristics of the sample image block.

[0027] Optionally, the weights corresponding to the multiple determination sub-models are determined according to the following steps:

[0028] Acquire image features of each of a plurality of test image blocks carrying labels, wherein the labels indicate whether the test image blocks are light spot generation areas;

[0029] Inputting the image features of each of the plurality of test image blocks carrying labels into the plurality of determination sub-models, and obtaining predicted probability values ​​of each of the plurality of test image blocks output by the plurality of determination sub-models being a light spot generation area;

[0030] Determine the accuracy of each of the determination sub-models according to the predicted probability values ​​of each of the multiple test image blocks output by each of the determination sub-models being a light spot generation area, and the labels carried by each of the multiple test image blocks;

[0031] The ratio of the accuracy of each determination sub-model to the total accuracy of the multiple determination sub-models is determined as the weight corresponding to the determination sub-model.

[0032] Optionally, inputting the image features of the plurality of image blocks into a light spot generation region determination model to obtain a binary classification result of whether each of the plurality of image blocks is a light spot generation region includes:

[0033] Inputting the image features of the plurality of image blocks into the plurality of determination sub-models respectively, to obtain probability prediction values ​​of the plurality of image blocks outputted by the plurality of determination sub-models as light spot generation areas;

[0034] For each of the image blocks, weighted summing is performed on the probability prediction values ​​of the image output by the multiple determination sub-models according to the weights corresponding to the multiple determination sub-models, so as to obtain a total probability prediction value of the image block being a light spot generation area;

[0035] According to the total probability prediction value that each of the plurality of image blocks is a light spot generation area, a binary classification result of whether each of the plurality of image blocks is a light spot generation area is obtained.

[0036] A second aspect of the embodiments of the present application provides a method for generating a light spot, the method comprising:

[0037] According to the method of the first aspect of the embodiment of the present application, determine the position of the light spot generation area in the image;

[0038] A light spot is generated in the determined light spot generation area to obtain an image with the light spot.

[0039] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method for determining a light spot generation area as described in the first aspect of an embodiment of the present application, or the method for generating a light spot as described in the second aspect of an embodiment of the present application.

[0040] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method for determining the light spot generation area described in the first aspect of the embodiments of the present application is implemented, or the method for generating the light spot described in the second aspect of the embodiments of the present application is implemented.

[0041] In a fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the method for determining the light spot generation area described in the first aspect of the embodiments of the present application, or the method for generating the light spot described in the second aspect of the embodiments of the present application.

[0042] The embodiments of the present application include the following advantages:

[0043] In this embodiment, the image to be detected can be cut into multiple image blocks, and the binary classification result of whether each image block is a light spot generation area can be obtained, so that the position of the light spot generation area in the entire image to be detected can be obtained according to the binary classification result of each image block and its position in the image to be detected; compared with analyzing and identifying the entire image to be detected, analyzing and identifying with image blocks as the granularity has the advantages of low processing difficulty and high accuracy. In addition, the binary classification result of the image block is determined by the light spot generation area determination model, and the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature. Therefore, it can accurately determine whether each image block is a light spot generation area, and then the accurate position of the light spot generation area in the image to be detected can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1 It is a light spot effect picture;

[0046] Figure 2 is a flowchart of the steps of a method for determining a light spot generation area in an embodiment of the present application;

[0047] Figure 3 is a flowchart of a method for generating a light spot in an embodiment of the present application;

[0048] Figure 4 It is a structural schematic diagram of a device for determining a light spot generation area in an embodiment of the present application;

[0049] Figure 5 is a structural schematic diagram of a light spot generating device in an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0052] In recent years, research on computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence has made important progress. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving many types of technologies such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks, etc. Computer vision, as an important branch of artificial intelligence, specifically allows machines to recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security control, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, automatic driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, and smart temperature measurement.

[0053] Reference Figure 2 As shown, a flowchart of a method for determining a light spot generation area in an embodiment of the present application is shown. Figure 2 As shown, the method for determining the light spot generation area can be used in electronic devices such as computers, mobile phones, tablet computers, servers, etc. The method for determining the light spot generation area includes the following steps:

[0054] Step S11: cutting the image to be detected into a plurality of image blocks, and recording the position of each image block in the image to be detected.

[0055] Step S12: extracting image features of the multiple image blocks;

[0056] Step S13: inputting the image features of the multiple image blocks into a light spot generation area determination model to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area, wherein the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature of the sample image block, and the label represents whether the sample image block is a light spot generation area;

[0057] Step S14: determining the position of the light spot generation area in the image to be detected according to the binary classification results of each of the multiple image blocks and their respective positions in the image to be detected.

[0058] The flare effect of an image is usually a light and shadow effect formed by blurring the highlight area in the image. When taking photos with a SLR, setting the aperture value to a large aperture can directly capture images with a flare effect. When taking photos with mobile phones and other devices, it is often difficult to directly capture images with a flare effect because the aperture is not large enough.

[0059] In the related art, when blurring an image, a light spot effect is usually generated based on a method similar to Gaussian filter convolution. However, when the image is processed by convolution, the light spots generated are often not in shape and are difficult to be bright, transparent and sharp-edged. In order to generate a clear light spot, it is first necessary to determine the location of the light spot generation area in the image, usually the location of the highlight point, and then a series of processing can be performed on the light spot generation area to generate a light spot effect similar to that obtained by SLR photography.

[0060] Taking into account the disadvantages of directly detecting the position of the light spot generation area of ​​the entire image to be detected, such as consuming a lot of computing resources and having low accuracy, the applicant proposes: equally cutting the image to be detected into a plurality of image blocks of equal area, and detecting whether each image block is a light spot generation area, so that the position of the light spot generation area in the image to be detected can be obtained based on the position of the image block of the light spot generation area in the image to be detected according to the detection result.

[0061] The image to be detected may be a clear image without light spots. To generate light spots on the image to be detected, it is first necessary to determine the position of the light spot generation area in the image to be detected.

[0062] It is understandable that there is no difference between images and image blocks in essence. Therefore, the spot generation area determination model can determine whether the image / image block corresponding to the image feature is a spot generation area based on the input image feature; however, the smaller the area of ​​the input image / image block, the more accurate the determination result; but the spot generation area determination model directly judges based on the image features of the entire image, and can only obtain a binary classification result, and cannot determine the position of the spot generation area in the entire image. Therefore, it is necessary to cut the image to be detected into multiple image blocks and record the position of each image block in the image to be detected. The position of each image block in the image to be detected can be recorded as coordinates or according to the serial number, and this application does not limit this.

[0063] The image features of the image block to be detected can include: color value features, gray value features, brightness value features, etc. The image feature extraction method can refer to the relevant technology. Among them, the gray value feature of the image block can be extracted by obtaining the image block after grayscale processing; it can be directly grayscale processing of each image block, or grayscale processing of the image to be detected, and then cutting the grayscale processed image to be detected into multiple grayscale processed image blocks; the grayscale processing method can be to extract the maximum color value of the R channel color value, G channel color value and B channel color value of the image / image block to be detected to form a single-channel grayscale image. The brightness value feature of the image block can be directly converted into a YUV image, and then the value of the Y channel of each image block is extracted, or the image to be detected can be converted into a YUV image, and then the image to be detected in the YUV format is cut into multiple image blocks.

[0064] By inputting the image features of multiple image blocks into the light spot generation area determination model, a binary classification result of whether each image block is a light spot generation area can be obtained. The light spot generation area determination model learns the relationship between the label carried by the sample image block and the image features of the sample image block, and the label represents whether the sample image block is a light spot generation area. The training method of the light spot generation area determination model will be introduced in detail later.

[0065] According to the binary classification result of each image block and the position of each image block in the image to be detected, the position of the light spot generation area in the image to be detected can be determined. For example, if the binary classification result of an image block is "is the light spot generation area", and the position of the image block in the image to be detected is the upper left corner area, it can be known that the upper left corner area of ​​the image to be detected (the area where the image block is located) is the light spot generation area.

[0066] By adopting the technical solution of the embodiment of the present application, the image to be detected can be cut into multiple image blocks, and the binary classification result of whether each image block is a light spot generation area can be obtained, so that the position of the light spot generation area in the entire image to be detected can be obtained according to the binary classification result of each image block and its position in the image to be detected; compared with analyzing and identifying the entire image to be detected, analyzing and identifying with image blocks as the granularity has the advantages of low processing difficulty and high accuracy. In addition, the binary classification result of the image block is determined by the light spot generation area determination model, and the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature. Therefore, it can accurately determine whether each image block is a light spot generation area, and then the accurate position of the light spot generation area in the image to be detected can be obtained.

[0067] Optionally, based on the above technical solution, in order to train the light spot generation area determination model, it is necessary to obtain a sample image block, and the sample image block can be obtained according to the following steps:

[0068] Step S21: Acquire a plurality of sample image pairs, each sample image pair comprising a sample image with light spots and a sample image without light spots having the same image content as the sample image with light spots.

[0069] Sample image pairs can be collected for the SLR and the fixed tripod. The sample images with light spots can be collected by setting the aperture value of the SLR to a large aperture (for example, f 2.8), and the sample images without light spots can be collected by setting the aperture value of the SLR to a small aperture (for example, f 16). In order to ensure the consistency of the image content of the sample image pairs, when collecting two sample images in the same sample image pair, the other parameters of the SLR are consistent, and the same scene is photographed. Among them, considering that the light spot effect often appears in the lighting scene at night, and the green plants, tree shade and other scenes during the day, the scenes for taking pictures are also mainly concentrated in the above scenes.

[0070] After obtaining paired sample images, the sample images can be screened to remove image pairs with fewer light sources, unclear light source points, and inconsistent image content. It is understandable that after a sample image is removed, the other sample image paired with it will also be removed accordingly.

[0071] Optionally, after screening the sample images, there may be 100 pairs of sample images left, and 100 sample images with light spots are organized into a sample set with light spots, and 100 sample images without light spots are organized into a sample set without light spots. The sample images in the sample set with light spots are named in sequence: with light spots-001, with light spots-002, ..., with light spots-099, with light spots-100; correspondingly, the sample image without light spots paired with with light spots-001 is named without light spots-001, and the sample image without light spots paired with with light spots-002 is named without light spots-002, ..., thereby obtaining a sample set without light spots composed of the named sample images without light spots.

[0072] Step S22: Select the light spot area of ​​the light spot sample image in each of the sample image pairs to obtain a sample image with a light spot position frame.

[0073] LabelImage (a labeling tool) can be used to label the spot area of ​​each sample image with a spot in the sample image set with a spot, obtain the spot position frame on each sample image with a spot, and mark each spot position frame, for example, mark the entire area occupied by each spot position frame in red.

[0074] Optionally, a file may be generated for each sample image with light spots, which records the name of the sample image with light spots, the serial number of each light spot position frame in the image with light spots, and the serial number and position information of each light spot position frame in the image with light spots, wherein the position information may be the values ​​of the coordinates x and y of the upper left corner of the light spot position frame and the width and height of the light spot position frame. Then, according to the file of each sample image with light spots, a binary mask map of the light spot area is generated for the sample image with light spots.

[0075] Specifically, for each sample image with light spots, an initial mask map with the same size as the sample image with light spots is generated, and each initial value in the initial mask map is 0; then the cv::rectangle function (a rectangular map drawing tool) in the opencv library (a computer vision) is used to take the position information of each light spot position frame as input to generate a binary mask map, and the area corresponding to the light spot position frame in the sample image with light spots in the binary mask map is white, and the rest is black. Therefore, a mask atlas can be generated, and each mask map in the mask atlas corresponds to each sample image with light spots in the sample image set with light spots. Therefore, each mask map in the mask atlas can also be named accordingly: mask-001, mask-002, ..., mask-099, mask-100. It can be understood that each mask image in the mask image set corresponds one-to-one to each sample image with light spots in the sample image set with light spots, and each sample image with light spots in the sample image set with light spots corresponds one-to-one to each sample image without light spots in the sample image set without light spots. Therefore, each mask image in the mask image set also corresponds one-to-one to each sample image without light spots in the sample image set without light spots.

[0076] Step S23: Cut the sample image with spot and the corresponding sample image without spot with the same granularity to obtain a plurality of sample image block pairs, each of which includes: a first sample image block derived from the sample image with the spot position frame, and a second sample image block derived from the sample image without spot, wherein the position of the first sample image block in the sample image with the spot position frame is the same as the position of the second sample image block in the sample image without spot, and the two sample image blocks in each sample image block pair are respectively derived from two images of the same sample image pair.

[0077] The rect function (a rectangular function) in the opencv library can be used to cut the sample image with the spot position frame and the corresponding sample image without the spot with the same granularity to obtain multiple sample image block pairs. For example, if the sample image with the spot is cut into 4×4 sample image blocks of equal size, the sample image without the spot is cut in the same way to obtain 4×4 sample image blocks of equal size. It can be understood that the finer the cutting granularity, the more accurate the trained model for determining the spot generation area.

[0078] Each sample image block pair is derived from two images belonging to the same sample image pair, one from the image with light spots and the other from the image without light spots, and the position of one of the sample image blocks in the original image is the same as the position of the other sample image block in the original image. For example, if a sample image block is derived from light spot-001, then the other sample image block belonging to the same sample image block pair as the sample image block is derived from without light spot-001, and the position of the sample image block in light spot-001 is the same as the position of the other sample image block in without light spot-001.

[0079] Optionally, the sample image can be cut horizontally and vertically according to 5% of the length and width of the original image, then each sample image can be cut into 400 sample image blocks, and the sample image blocks with light spot samples can be named in sequence from upper left to lower right: light spot-001-001, light spot-001-002, ... light spot-001-399, light spot-001-400, ..., light spot-100-001, light spot-100-002, ... light spot-100-399, light spot-100-400. Correspondingly, the sample image blocks of the no-spot sample can also be named in sequence: no-spot-001-001, no-spot-001-002, ... no-spot-001-399, no-spot-001-400, ..., no-spot-100-001, no-spot-100-002, ... no-spot-100-399, no-spot-100-400.

[0080] Optionally, the mask image can be cut with the same granularity according to the same idea to obtain multiple mask image blocks: mask-001-001, mask-001-002, ... mask-001-399, mask-001-400, ..., mask-100-001, mask-100-002, ... mask-100-399, mask-100-400. It can be understood that the mask image blocks, the sample image blocks of the samples without light spots, and the sample image blocks of the samples with light spots are also one-to-one corresponding.

[0081] Step S24: determining whether the first sample image block belongs to the area where the light spot is located according to the ratio of the area belonging to the light spot position frame in each first sample image block to the total area of ​​the first sample image block.

[0082] It is understandable that the second image block cut out from the sample image without light spots must not contain light spots, while some of the first image blocks cut out from the sample image with light spots contain light spots and some do not, and in the first image blocks containing light spots, only a small part of the area may be light spots. Therefore, the first sample image blocks derived from the sample image with the light spot position frame can be screened.

[0083] Whether the first sample image block belongs to the light spot location area is determined by the ratio of the area belonging to the light spot location frame in the first sample image block to the total area of ​​the first sample image block. Generally, the first sample image block whose area belonging to the light spot location frame accounts for more than 30% of the total area can be determined as the light spot location area.

[0084] Optionally, it is possible to determine whether the first sample image block belongs to the area where the light spot is located by judging the ratio of the area marked in red in the first sample image block to the total area of ​​the first sample image block. Optionally, it is also possible to determine whether the first sample image block belongs to the area where the light spot is located by judging the ratio of the area of ​​the white area in the mask image block corresponding to the first sample image block to the total area of ​​the mask image block. Wherein, directly judging by the area marked in red in the first sample image block may result in an inaccurate result of determining whether the first sample image block belongs to the area where the light spot is located because there is a red area in the first sample image block itself; judging the area where the light spot is located by the mask image block can avoid this problem, thereby obtaining a more accurate result.

[0085] It is understandable that the number of image blocks in the first sample image blocks belonging to the area where the light spot is located is usually much smaller than the number of image blocks that do not belong to the area where the light spot is located. Therefore, the number of the two types of first sample image blocks can be balanced, and some first sample image blocks that do not belong to the area where the light spot is located can be randomly eliminated, so as to obtain a balanced number of first sample image block sets in the light spot area and first sample image block sets in the non-light spot area.

[0086] Correspondingly, the second sample image blocks and mask image blocks corresponding to the eliminated first sample image blocks are also eliminated, so as to obtain two second sample image block sets and two mask image block sets with balanced numbers, wherein one of the two second sample image block sets is a data set corresponding to the first sample image block set in the spot area, and the other second sample image block set is a data set corresponding to the first sample image block set in the non-spot area; and one of the two mask image block sets is a data set corresponding to the first sample image block set in the spot area, and the other mask image block set is a data set corresponding to the first sample image block set in the non-spot area.

[0087] Step S25: determining the sample image block pair to which the first sample image block belonging to the area where the light spot is located belongs as the light spot sample image block pair;

[0088] Step S26: determining the second sample image block belonging to the spot sample image block pair as a sample image block carrying a positive label;

[0089] Step S27: Determine the second sample image block that does not belong to the spot sample image block pair as a sample image block carrying a negative label.

[0090] Optionally, the sample image block pair to which the first sample image block belonging to the area where the light spot is located belongs can be determined as a light spot sample image block pair; then the second sample image block in the sample image block pair should be an image block that can generate a light spot after processing. Therefore, the second sample image block belonging to the light spot sample image block pair can be determined as a sample image block carrying a positive label, and the positive label indicates that the second sample image block is a light spot generation area. The second sample image block that does not belong to the light spot sample image block pair is determined as a sample image block carrying a negative label, and the negative label indicates that the second sample image block is not a light spot generation area.

[0091] Optionally, each second sample image block in the second sample image block set corresponding to the first sample image block set in the spot area can be directly determined as a sample image block carrying a positive label; each second sample image block in the second sample image block set corresponding to the first sample image block set in the non-spot area can be determined as a sample image block carrying a negative label. Optionally, it can also be determined whether the second sample image block in each second sample image block set should carry a positive label or a negative label based on the mask image block set.

[0092] The content of the sample image without light spots and the sample image with light spots in the sample image pair is consistent, so if you want to generate a light spot in the sample image without light spots, the position of the generated light spot should be the same as the position of the light spot in the sample image with light spots. Therefore, the light spot information in the sample image with light spots can be used as the label information of whether each image block in the sample image without light spots is a light spot generation area. For this reason, by performing multiple processing on the sample image without light spots and the sample image with light spots as in the technical solution of the embodiment of the present application, the sample image blocks used for training the light spot generation area determination model can be obtained.

[0093] Optionally, based on the above technical solution, a sample image block used for training the light spot generation area determination model is obtained, and the sample image block can be used to perform supervised training on the initial model to obtain a trained light spot generation area determination model.

[0094] The image features of the sample image blocks (sample image blocks carrying positive labels and sample image blocks carrying negative labels) used by each training spot generation area determination model are extracted. The method for extracting the image features of the sample image blocks can refer to the method for extracting the image features of the image blocks of the image to be detected in the previous text.

[0095] The image features of each sample image block are input into the initial model to obtain the binary classification prediction results of each sample image block output by the initial model as to whether it is a light spot generation area; a loss function is established according to the difference between the binary classification prediction results of each sample image block and the label carried by the sample image block; the model parameters of the initial model are updated based on the loss function, and when the initial model converges, a light spot generation area determination model is obtained.

[0096] In this way, the trained light spot generation area determination model can learn the relationship between the label carried by the sample image block and the image feature of the sample image block, and thus determine whether the image / image block corresponding to the image feature is a light spot area detection model based on the image feature.

[0097] Optionally, based on the above technical solution, the light spot generation area determination model may include multiple determination sub-models, and the probability of the image block being the light spot generation area is determined according to each sub-model, and a binary classification result is output. The determination sub-model may be a support vector machine, a decision tree, etc.

[0098] In the embodiment of the present application, the light spot generation area determination model is obtained by fusing multiple determination sub-models according to their respective corresponding weights, and each determination sub-model is obtained by using the image features of the sample image block and the label carried by the sample image block to perform supervised training on the initial sub-model. In other words, the light spot generation area determination model is obtained by training the light spot generation area determination model including multiple determination sub-models using sample image blocks. Specifically, it may include: extracting the image features of each sample image block used to train the light spot generation area determination model; according to the image features of each sample image block and the label carried by the sample image block, respectively performing supervised training on multiple initial sub-models to obtain multiple determination sub-models; obtaining the weights corresponding to each of the multiple determination sub-models; fusing the multiple determination sub-models according to their respective corresponding weights to obtain the light spot generation area determination model.

[0099] Among them, supervised training of an initial sub-model can be as follows: obtaining the probability prediction value of the sample image block output by the initial sub-model according to the image features of the sample image block as a light spot generation area; establishing a loss function based on the difference between the probability prediction value of the sample image block and the label of the sample image block, and updating the model parameters of the initial sub-model based on the loss function to obtain a trained determination sub-model.

[0100] The weight corresponding to each determination sub-model is obtained, and the multiple determination sub-models are fused according to the corresponding weights to obtain the light spot generation area determination model.

[0101] Optionally, on the basis of the above technical solution, obtaining the weight corresponding to each determination sub-model may include: obtaining image features of each of a plurality of test image blocks carrying labels, the label indicating whether the test image block is a light spot generation area, a positive label indicating that the test image block is a light spot generation area, and a negative label indicating that the test image block is not a light spot generation area, wherein the method for obtaining the test image block may refer to the method for obtaining the sample image block; inputting the image features of each of the plurality of test image blocks carrying labels into a plurality of determination sub-models to obtain predicted probability values ​​of each of the plurality of test image blocks output by the plurality of determination sub-models being a light spot generation area; determining the accuracy of each determination sub-model according to the predicted probability values ​​of each of the plurality of test image blocks output by each determination sub-model being a light spot generation area, and the labels carried by each of the plurality of test image blocks; determining the ratio of the accuracy of each determination sub-model to the total accuracy of the plurality of determination sub-models as the weight corresponding to the determination sub-model.

[0102] For example, for a determination sub-model, the prediction probability of a test image block carrying a positive label three times being a light spot generation area is 0.9, the prediction probability of a test image block carrying a positive label twice being a light spot generation area is 0.8, the prediction probability of a test image block carrying a negative label four times being a light spot generation area is 0.4, and the prediction probability of a test image block carrying a negative label once being a light spot generation area is 0.8. Then, the accuracy of the determination sub-model can be ((3×0.9)+(2×0.8)+(4×0.6)+(1×0.2)) / 10=0.069.

[0103] Optionally, each determination sub-model may also output a binary classification result, and when the predicted probability value of the test image block is greater than a preset threshold, the output binary classification result indicates that the test image block is a light spot generation area. Therefore, using the previous example, if the preset threshold is 0.7, the accuracy of the determination sub-model is (3×100%+2×100%+4×100%+1×0%) / 10=0.9.

[0104] According to the ratio of the accuracy of each determination sub-model to the total accuracy of multiple determination sub-models, the weight corresponding to the determination sub-model can be obtained.

[0105] Optionally, when the light spot generation area determination model includes multiple determination sub-models, the image features of multiple image blocks are input into the light spot generation area determination model to obtain the binary classification results of whether each of the multiple image blocks is a light spot generation area, including: inputting the image features of the multiple image blocks into the multiple determination sub-models respectively to obtain the probability prediction values ​​of the multiple image blocks output by the multiple determination sub-models as light spot generation areas; for each image block, according to the weights corresponding to each of the multiple determination sub-models, weighted summing the probability prediction values ​​of the image output by the multiple determination sub-models to obtain the total probability prediction value of the image block as a light spot generation area; according to the total probability prediction values ​​of the multiple image blocks as light spot generation areas, obtaining the binary classification results of whether each of the multiple image blocks is a light spot generation area. Wherein, for each image block, when the total probability prediction value of the image block as a light spot generation area is greater than the light spot probability threshold, the binary classification result of whether the image block is a light spot generation area is determined to be yes; when the total probability prediction value of the image block as a light spot generation area is not greater than the light spot probability threshold, the binary classification result of whether the image block is a light spot generation area is determined to be no.

[0106] For example, the light spot generation area determination model includes three determination sub-models, and the corresponding weights of the three determination sub-models are 0.3, 0.3, and 0.4 respectively, and the light spot probability threshold is 0.7; the three determination sub-models predict that the probability prediction values ​​of the same image block as the light spot generation area are 0.7, 0.8, and 0.9 respectively, then the total probability prediction value of the image block is 0.3×0.7+0.3×0.8+0.4×0.9=0.81; because 0.81>0.7, the binary classification result of the image block output by the light spot generation area determination model is "the image block is the light spot generation area".

[0107] By adopting the technical solution of the embodiment of the present application, the binary classification detection result finally output by the light spot generation area model integrates the prediction results of multiple determination sub-models on whether the image block is a light spot generation area, and the corresponding weights of the multiple determination sub-models are determined according to the accuracy rate. Compared with detection by only one model, it has higher accuracy.

[0108] Optionally, based on the above technical solution, one of the multiple determination sub-models included in the light spot generation area determination model can be a color value model, and the color value model is obtained by using the first predicted probability value of the sample image block and the label carried by the sample image block to perform supervised training on the initial color value model, and the first predicted probability value represents: the initial color value model predicts the probability value of the sample image block as the light spot generation area based on the color value feature of the sample image block. The determination sub-model mainly makes judgments based on the color value features of the image. The supervised training of the determination sub-model can be:

[0109] The color value features of the sample image blocks used by each training spot generation area determination model are extracted, and the color value features are RGB features. Usually, the sample image block is an RGB image, so the RGB features of the sample image block can be directly extracted. If the sample image block is not an RGB image, the sample image block can be converted into an RGB image, and then the RGB features can be extracted. Optionally, the RGB features of the sample image block can be represented by a vector, and the vector is normalized and cleaned to remove the influence of outliers, and the missing values ​​are filled with adjacent values.

[0110] The color value feature of the sample image block is input into the initial color value model to obtain a first predicted probability value that the sample image block is a light spot generation area; according to the first predicted probability value of the sample image block and the label carried by the sample image block, the initial color value model is supervised trained to obtain a color value model.

[0111] Optionally, based on the above technical solution, one of the multiple determination sub-models included in the light spot generation area determination model can be a gray value model, which is obtained by using the second predicted probability value of the sample image block and the label carried by the sample image block to perform supervised training on the initial gray value model, and the second predicted probability value represents: the initial gray value model predicts the probability value of the sample image block as the light spot generation area based on the gray value feature of the sample image block. The determination sub-model mainly makes judgments based on the gray value features of the image. The supervised training of the determination sub-model can be:

[0112] Obtaining the grayscale value features of the sample image block specifically includes: obtaining the maximum color value of the color values ​​of the three channels of the R channel color value, the G channel color value and the B channel of the sample image block, using the maximum color value to generate a grayscale image of the sample image block, and then extracting the grayscale value features in the grayscale image. Optionally, it may also include: when generating a sample image block carrying a label, directly generating a grayscale image of a spot-free sample image, then cutting the grayscale image, labeling the image blocks, etc., and then extracting the grayscale value features of each sample image block. Among them, the grayscale image cutting, the image block labeling, etc. are the same as the method of cutting the spot-free sample image, the image block labeling, etc. In this way, compared with obtaining the grayscale image of each small sample image block, directly obtaining the grayscale image of the spot-free sample image can save computing resources.

[0113] Optionally, the grayscale value features of the sample image block can be represented by a vector, and the vector data can be normalized and cleaned to remove the influence of outliers and fill in the missing values ​​with adjacent values.

[0114] The gray value feature of the sample image block is input into the initial color value model to obtain a second predicted probability value that the sample image block is a light spot generation area; according to the second predicted probability value of the sample image block and the label carried by the sample image block, the initial gray value model is supervised trained to obtain a gray value model.

[0115] By adopting the technical solution of the embodiment of the present application, a grayscale image of a sample image block is generated according to the maximum value of the RGB three channels. Compared with generating a grayscale image of a sample image block according to methods such as the average value of the RGB three channels, the highlight area and the non-highlight area in the image can be more clearly distinguished. Therefore, the light spot generation area is usually the highlight area. Therefore, the grayscale image generated by the method of the embodiment of the present application, the grayscale value model obtained by training is more accurate.

[0116] Optionally, based on the above technical solution, one of the multiple determination sub-models included in the light spot generation area determination model can be a brightness value model, which is obtained by using the third predicted probability value of the sample image block and the label carried by the sample image block to perform supervised training on the initial brightness value model, and the third predicted probability value represents: the initial brightness value model predicts the probability value of the sample image block as the light spot generation area based on the brightness value feature of the sample image block. The determination sub-model mainly makes judgments based on the brightness value features of the image. The supervised training of the determination sub-model can be:

[0117] Obtaining the brightness value feature of the sample image block specifically includes: converting the RGB of the sample image block into the YUV format to obtain the sample image block in the YUV format, and extracting the brightness value feature contained in the Y channel of the sample image block in the YUV format. Optionally, it may also include: when generating the sample image block carrying the label, directly converting the spot-free sample image into the YUV format, and then converting the sample image into the YUV format for cutting, labeling the image blocks, etc., and then extracting the brightness value feature of each sample image block in the YUV format. The cutting of the sample image in the YUV format, labeling of the image blocks, etc. are the same as the cutting of the spot-free sample image, labeling of the image blocks, etc. In this way, compared with converting each small sample image block into the YUV format, directly converting the spot-free sample image into the YUV format can save computing resources.

[0118] Optionally, a vector may be used to represent the brightness value feature of the sample image block, and the vector may be normalized and cleaned to remove the influence of outliers and fill in the missing values ​​with adjacent values.

[0119] The brightness value feature of the sample image block is input into the initial color value model to obtain a third predicted probability value that the sample image block is a light spot generation area; according to the third predicted probability value of the sample image block and the label carried by the sample image block, the initial brightness value model is supervisedly trained to obtain a brightness value model.

[0120] By adopting the technical solution of the embodiment of the present application, considering that the light spot generation area is usually a highlight area, the model is trained using the brightness value characteristics of the sample image block. The trained brightness value model can determine whether the image block is a light spot generation area from the brightness aspect.

[0121] The light spot generation area determination model combines the above color value model, gray value model, and brightness value model to determine whether an image block is a light spot generation area from multiple dimensions. Therefore, the light spot generation area determination model has high accuracy.

[0122] Reference Figure 3 As shown, a flow chart of the steps of a method for generating a light spot in an embodiment of the present application is shown. Figure 3As shown, the light spot generation method can be used in electronic devices such as computers, mobile phones, tablet computers, servers, etc. The light spot generation method includes the following steps:

[0123] S21: determining the position of the light spot generation area in the image according to the light spot generation area determination method;

[0124] S22: generating a light spot on the determined light spot generation area to obtain an image with the light spot.

[0125] The image is an image to which a light spot is added, and specifically may be an image without a light spot. The method for determining a light spot generation area may be the method for determining a light spot generation area provided in an embodiment of the present application.

[0126] For an image to which a light spot needs to be added, the light spot generation area determination method provided in the embodiment of the present application can be used to determine the position of the light spot generation area in the image, and then a bright and clear light spot is added at the determined position to obtain an image with a light spot. The specific method of adding a light spot can be: pre-acquire a picture of a standard light spot (for example: a picture of a light spot that is considered bright and clear by the human eye), and then overlay the picture of the standard light spot on the determined position in the image.

[0127] By adopting the light spot generation method provided in the embodiment of the present application, a light spot is generated at an accurate position, the visual effect of the light spot is improved, and an image with a bright light spot with clear edges is obtained.

[0128] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0129] Figure 4 is a schematic diagram of the structure of a device for determining a light spot generation area according to an embodiment of the present application. Figure 4 As shown, the light spot generation area determination device includes an image cutting module, a feature extraction module, a result acquisition module and a position determination module, wherein:

[0130] An image cutting module is used to cut the image to be detected into multiple image blocks and record the position of each image block in the image to be detected;

[0131] A feature extraction module, used to extract image features of the multiple image blocks;

[0132] A result acquisition module is used to input the image features of the multiple image blocks into a light spot generation area determination model to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area, wherein the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature of the sample image block, and the label represents whether the sample image block is a light spot generation area;

[0133] The position determination module is used to determine the position of the light spot generation area in the image to be detected according to the binary classification results of each of the multiple image blocks and their respective positions in the image to be detected.

[0134] Optionally, the sample image blocks used for training the light spot generation area determination model are obtained according to the following steps:

[0135] Acquire a plurality of sample image pairs, each sample image pair comprising a sample image with light spots and a sample image without light spots that has the same image content as the sample image with light spots;

[0136] Selecting a light spot region of the light spot sample image in each of the sample image pairs to obtain a sample image with a light spot position frame;

[0137] The sample image with light spot having the light spot position frame and the corresponding sample image without light spot are cut with the same granularity to obtain a plurality of sample image block pairs, each of which comprises: a first sample image block derived from the sample image with the light spot position frame, and a second sample image block derived from the sample image without light spot, wherein the position of the first sample image block in the sample image with the light spot position frame is the same as the position of the second sample image block in the sample image without light spot, and the two sample image blocks in each sample image block pair are derived from two images of the same sample image pair respectively;

[0138] Determine whether the first sample image block belongs to the area where the light spot is located according to the ratio of the area belonging to the light spot position frame in each first sample image block to the total area of ​​the first sample image block;

[0139] Determine the sample image block pair to which the first sample image block belonging to the area where the light spot is located as the light spot sample image block pair;

[0140] Determine the second sample image block belonging to the spot sample image block pair as a sample image block carrying a positive label;

[0141] The second sample image block that does not belong to the spot sample image block pair is determined as a sample image block carrying a negative label.

[0142] Optionally, the step of training the light spot generation area determination model includes:

[0143] Extracting image features of each sample image block used for training the light spot generation area determination model;

[0144] Inputting the image features of each sample image block into the initial model to obtain a binary classification prediction result of whether each sample image block is a light spot generation area;

[0145] Establishing a loss function according to the difference between the binary classification prediction result of each sample image block and the label carried by the sample image block;

[0146] The model parameters of the initial model are updated based on the loss function to obtain the light spot generation area determination model.

[0147] Optionally, the light spot generation area determination model is obtained by fusing multiple determination sub-models according to their respective weights, and each of the determination sub-models is obtained by supervised training of an initial sub-model using image features of a sample image block and a label carried by the sample image block.

[0148] Optionally, one of the multiple determination sub-models is a color value model, which is obtained by supervised training of an initial color value model using a first predicted probability value of a sample image block and a label carried by the sample image block, and the first predicted probability value represents: the initial color value model predicts the probability value of the sample image block as a light spot generation area based on the color value characteristics of the sample image block.

[0149] Optionally, one of the multiple determination sub-models is a gray value model, which is obtained by supervised training of an initial gray value model using a second predicted probability value of the sample image block and a label carried by the sample image block, and the second predicted probability value represents: the initial gray value model predicts the probability value of the sample image block as a light spot generation area based on the gray value characteristics of the sample image block.

[0150] Optionally, one of the multiple determination sub-models is a brightness value model, which is obtained by supervised training of an initial brightness value model using a third predicted probability value of the sample image block and a label carried by the sample image block, and the third predicted probability value represents: the initial brightness value model predicts the probability value of the sample image block as a light spot generation area based on the brightness value characteristics of the sample image block.

[0151] Optionally, the weights corresponding to the multiple determination sub-models are determined according to the following steps:

[0152] Acquire image features of each of a plurality of test image blocks carrying labels, wherein the labels indicate whether the test image blocks are light spot generation areas;

[0153] Inputting the image features of each of the plurality of test image blocks carrying labels into the plurality of determination sub-models, and obtaining predicted probability values ​​of each of the plurality of test image blocks output by the plurality of determination sub-models being a light spot generation area;

[0154] Determine the accuracy of each of the determination sub-models according to the predicted probability values ​​of each of the multiple test image blocks output by each of the determination sub-models being a light spot generation area, and the labels carried by each of the multiple test image blocks;

[0155] The ratio of the accuracy of each determination sub-model to the total accuracy of the multiple determination sub-models is determined as the weight corresponding to the determination sub-model.

[0156] Optionally, the result acquisition module includes:

[0157] A sub-model prediction unit, used to input the image features of the multiple image blocks into the multiple determination sub-models respectively, to obtain probability prediction values ​​of the multiple image blocks output by the multiple determination sub-models as light spot generation areas;

[0158] a total probability acquisition unit, configured to perform weighted summation of the probability prediction values ​​of the image output by the multiple determination sub-models according to the respective weights of the multiple determination sub-models for each of the image blocks, so as to obtain a total probability prediction value of the image block being a light spot generation area;

[0159] The result determination unit is used to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area according to the total probability prediction value of each of the multiple image blocks being a light spot generation area.

[0160] Figure 5 is a schematic diagram of the structure of a light spot generating device according to an embodiment of the present application. Figure 5 As shown, the light spot generating device includes a position determining module and a light spot generating module, wherein:

[0161] The position determination module is used to determine the position of the light spot generation area in the image according to the light spot generation area determination method provided in the embodiment of the present application;

[0162] The light spot generation module is used to generate a light spot on the determined light spot generation area to obtain an image with a light spot.

[0163] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.

[0164] The present application also provides an electronic device, referring to Figure 6 , Figure 6 Schematic diagram of an electronic device proposed in an embodiment of the present application. Figure 6 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. A computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement the steps in the spot generation area determination method or the spot generation method disclosed in the embodiment of the present application.

[0165] The embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method for determining a light spot generation area or the method for generating a light spot as disclosed in the embodiment of the present application is implemented.

[0166] The embodiment of the present application further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the light spot generation area determination method or light spot generation method as disclosed in the embodiment of the present application.

[0167] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0168] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0172] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0173] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0174] The above is a detailed introduction to a method for determining a light spot generation area, a light spot generation method, a device, a medium and a program product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for determining a light spot generation area, characterized in that: include: Cutting the image to be detected into multiple image blocks, and recording the position of each image block in the image to be detected; extracting image features of the plurality of image blocks; The image features of the multiple image blocks are input into a light spot generation area determination model to obtain a binary classification result of whether each of the multiple image blocks is a light spot generation area, wherein the light spot generation area determination model learns the relationship between the label carried by the sample image block and the image feature of the sample image block, and the label represents whether the sample image block is a light spot generation area; the sample image block includes a second sample image block carrying a positive label and a second sample image block carrying a negative label; the positive label indicates that the second sample image block belongs to a light spot sample image block pair, and the negative label indicates that the second sample image block does not belong to a light spot sample image block pair; the second sample image block is derived from a sample image without a light spot; the light spot sample image block pair refers to a sample image block pair to which a first sample image block belonging to an area where a light spot is located belongs; the area where the light spot is located is determined according to the ratio of the area belonging to a light spot position frame in each first sample image block to the total area of ​​the first sample image block; the first sample image block is derived from a sample image with a light spot position frame; The position of the light spot generation area in the image to be detected is determined according to the binary classification results of each of the multiple image blocks and their respective positions in the image to be detected.

2. The method according to claim 1, characterized in that The sample image blocks used to train the light spot generation area determination model are obtained according to the following steps: Acquire a plurality of sample image pairs, each sample image pair comprising a sample image with light spots and a sample image without light spots that has the same image content as the sample image with light spots; Selecting a light spot region of the light spot sample image in each of the sample image pairs to obtain a sample image with a light spot position frame; The sample image with light spot having the light spot position frame and the corresponding sample image without light spot are cut with the same granularity to obtain a plurality of sample image block pairs, each of which comprises: a first sample image block derived from the sample image with the light spot position frame, and a second sample image block derived from the sample image without light spot, wherein the position of the first sample image block in the sample image with the light spot position frame is the same as the position of the second sample image block in the sample image without light spot, and the two sample image blocks in each sample image block pair are derived from two images of the same sample image pair respectively; Determine whether the first sample image block belongs to the area where the light spot is located according to the ratio of the area belonging to the light spot position frame in each first sample image block to the total area of ​​the first sample image block; Determine the sample image block pair to which the first sample image block belonging to the area where the light spot is located as the light spot sample image block pair; Determine the second sample image block belonging to the spot sample image block pair as a sample image block carrying a positive label; The second sample image block that does not belong to the spot sample image block pair is determined as a sample image block carrying a negative label.

3. The method according to claim 2, characterized in that The training step of the light spot generation area determination model includes: Extracting image features of each sample image block used for training the light spot generation area determination model; Inputting the image features of each sample image block into the initial model to obtain a binary classification prediction result of whether each sample image block is a light spot generation area; Establishing a loss function according to the difference between the binary classification prediction result of each sample image block and the label carried by the sample image block; The model parameters of the initial model are updated based on the loss function to obtain the light spot generation area determination model.

4. The method according to claim 2, characterized in that: The light spot generation area determination model is obtained by fusing multiple determination sub-models according to their corresponding weights, and each of the determination sub-models is obtained by supervised training of the initial sub-model using the image features of the sample image block and the label carried by the sample image block.

5. The method according to claim 4, characterized in that One of the multiple determination sub-models is a color value model, and the color value model is obtained by supervised training of an initial color value model using a first predicted probability value of a sample image block and a label carried by the sample image block, and the first predicted probability value represents: the initial color value model predicts the probability value of the sample image block as a light spot generation area based on the color value characteristics of the sample image block.

6. The method according to claim 4, characterized in that One of the multiple determination sub-models is a gray value model, and the gray value model is obtained by supervised training of an initial gray value model using a second predicted probability value of the sample image block and a label carried by the sample image block, and the second predicted probability value represents: the initial gray value model predicts the probability value of the sample image block as a light spot generation area based on the gray value characteristics of the sample image block.

7. The method according to claim 4, characterized in that One of the multiple determination sub-models is a brightness value model; the brightness value model is obtained by supervised training of an initial brightness value model using a third predicted probability value of a sample image block and a label carried by the sample image block, and the third predicted probability value represents: the initial brightness value model predicts a probability value of the sample image block as a light spot generation area based on the brightness value characteristics of the sample image block.

8. The method according to any one of claims 4 to 7, characterized in that: The weights corresponding to the multiple determination sub-models are determined according to the following steps: Acquire image features of each of a plurality of test image blocks carrying labels, wherein the labels indicate whether the test image blocks are light spot generation areas; Inputting the image features of each of the plurality of test image blocks carrying labels into the plurality of determination sub-models, and obtaining predicted probability values ​​of each of the plurality of test image blocks output by the plurality of determination sub-models being a light spot generation area; Determine the accuracy of each of the determination sub-models according to the predicted probability values ​​of each of the multiple test image blocks output by each of the determination sub-models being a light spot generation area, and the labels carried by each of the multiple test image blocks; The ratio of the accuracy of each determination sub-model to the total accuracy of the multiple determination sub-models is determined as the weight corresponding to the determination sub-model.

9. The method according to any one of claims 4 to 7, characterized in that: Inputting the image features of the plurality of image blocks into a light spot generation area determination model to obtain a binary classification result of whether each of the plurality of image blocks is a light spot generation area includes: Inputting the image features of the plurality of image blocks into the plurality of determination sub-models respectively, and obtaining probability prediction values ​​of the plurality of image blocks outputted by the plurality of determination sub-models as light spot generation areas; For each of the image blocks, weighted summing is performed on the probability prediction values ​​of the image output by the multiple determination sub-models according to the weights corresponding to the multiple determination sub-models, so as to obtain a total probability prediction value of the image block being a light spot generation area; According to the total probability prediction value that each of the plurality of image blocks is a light spot generation area, a binary classification result of whether each of the plurality of image blocks is a light spot generation area is obtained.

10. A method for generating a light spot, characterized in that: The method comprises: Determining the position of the light spot generation area in the image according to any one of the methods of claims 1 to 9; A light spot is generated in the determined light spot generation area to obtain an image with the light spot.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the light spot generation area determination method according to any one of claims 1 to 9, or the light spot generation method according to claim 10.

12. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method for determining a light spot generation area according to any one of claims 1 to 9 or the method for generating a light spot according to claim 10 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for determining a light spot generation area according to any one of claims 1 to 9 or the method for generating a light spot according to claim 10 is implemented.

Images