Image correction method and device, computer device and storage medium

Abstract

This application relates to an image correction method, apparatus, computer device, storage medium, and computer program product. The method includes: detecting positioning symbols in an image to be processed to obtain the position of the positioning symbols in the image and the orientation represented by the positioning symbols; when the number of detected positioning symbols meets a preset condition, and the position corresponds to the orientation, connecting the positioning symbols to obtain a target region of the positioning symbols; and correcting the target region to obtain a corrected target image. This method can improve the accuracy of target region recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image correction method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In applications such as video conferencing, live online courses, and recorded courses, one type of user need is to ensure the display effect of whiteboard content during display / recording. In traditional technology, a dedicated camera is installed above the whiteboard, and edge detection is performed on the captured image to identify the whiteboard in the captured image.

[0003] However, this method can only probabilistically identify targets with clear boundaries (such as whiteboards), and the detection results are strongly correlated with edge clarity and edge detection algorithm. When there are no boundaries, the boundaries are not clear enough, or the boundaries are partially occluded, the identified target area will be incomplete and unclear. Correcting the target area will result in inaccurate image correction. Summary of the Invention

[0004] Therefore, it is necessary to provide an image correction method, apparatus, computer device, computer-readable storage medium, and computer program product that can more accurately determine the target area, addressing the aforementioned technical problems.

[0005] Firstly, this application provides an image correction method. The method includes:

[0006] The image to be processed is subjected to location symbol detection to obtain the position of the location symbol in the image and the orientation represented by the location symbol;

[0007] When the number of the location symbols meets the preset conditions and the location corresponds to the orientation, the location symbols are connected to obtain the target area of ​​the location symbols.

[0008] The target region is corrected to obtain the corrected target image.

[0009] In one embodiment, the step of performing location symbol detection on the acquired image to obtain the position of the location symbol in the image and the orientation represented by the location symbol includes:

[0010] A detection model for acquiring location symbols in response to target events;

[0011] The location symbols in the image are detected according to the detection model to obtain the position of the location symbols in the image and the features of the location symbols;

[0012] Based on the characteristics of the positioning symbol, the orientation represented by the positioning symbol is obtained.

[0013] In one embodiment, connecting the positioning symbols to obtain the target area of ​​the positioning symbols includes:

[0014] Connect the positioning symbols to obtain the initial area;

[0015] Determine whether the boundary of the initial region matches the boundary of the preset shape;

[0016] If not, then the initial region is filtered out;

[0017] If so, the initial region is filtered based on the boundary angle to determine whether the initial region is the target region.

[0018] In one embodiment, filtering the initial region based on the boundary angle to determine whether the initial region is a target region includes:

[0019] Calculate the included angle of the boundary of the initial region;

[0020] The initial region is filtered according to whether the included angle of the boundary is within the included angle range of the preset shape;

[0021] The filtered initial region is used as the target region.

[0022] In one embodiment, the positioning symbol includes a first positioning symbol and a second positioning symbol, wherein the first positioning symbol is a detected positioning symbol, the second positioning symbol is an undetected positioning symbol, and the position and orientation are the position and orientation of the first positioning symbol; the method further includes:

[0023] When the number of the first positioning symbols does not meet the preset condition, the detection area of ​​the second positioning symbol is determined according to the orientation;

[0024] In the detection area, the second positioning symbol is detected to obtain the updated number, updated position, and updated orientation of the first positioning symbol;

[0025] When the updated quantity meets the preset condition and the updated position is consistent with the updated orientation, the updated positioning symbols are connected to obtain the target area of ​​the positioning symbols.

[0026] In one embodiment, determining the detection area of ​​the second positioning symbol based on the orientation represented by the first positioning symbol includes:

[0027] Adjust the acquisition parameters of the shooting device according to the orientation represented by the first positioning symbol;

[0028] Based on the adjusted acquisition parameters, reacquire the image;

[0029] In the re-acquired image, the detection area of ​​the second positioning symbol is determined based on the orientation.

[0030] In one embodiment, correcting the target region to obtain a target image includes:

[0031] Generate the corresponding perspective transformation matrix according to the vertex positions of the preset shape;

[0032] The target region is corrected according to the perspective transformation matrix to obtain the corrected target image.

[0033] Secondly, this application also provides an image correction apparatus. The apparatus includes:

[0034] The detection module performs location symbol detection on the image to be processed, and obtains the position of the location symbol in the image and the orientation represented by the location symbol;

[0035] The target area determination module is used to connect the positioning symbols to obtain the target area of ​​the positioning symbols when the number of detected positioning symbols meets a preset condition and the position corresponds to the orientation.

[0036] The image correction module is used to correct the target area to obtain the corrected target image.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the image correction steps in any of the above embodiments.

[0038] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the image correction steps in any of the above embodiments.

[0039] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the image correction steps in any of the above embodiments.

[0040] The aforementioned image correction method, apparatus, computer equipment, storage medium, and computer program product perform location symbol detection on the image to be processed, obtaining the position of the location symbol in the image and the orientation represented by the location symbol; when the number of detected location symbols meets a preset condition, and the position corresponds to the orientation, the location symbols are connected to obtain the target region of the location symbol. The target region determined based on the location symbol is the region of interest, eliminating the need for edge detection algorithms to identify the target region, reducing the possibility of misidentification, achieving high accuracy, and obtaining the corrected target image by correcting the target region. Attached Figure Description

[0041] Figure 1 This is a diagram illustrating the application environment of an image correction method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating an image correction method in one embodiment;

[0043] Figure 3 This is a schematic diagram of the structure of a positioning symbol in one embodiment;

[0044] Figure 4 This is a schematic diagram of the structure of a positioning symbol in one embodiment;

[0045] Figure 5 This is a schematic diagram illustrating an application scenario of a traditional image correction method.

[0046] Figure 6 A schematic diagram illustrating an application scenario for another traditional image correction method;

[0047] Figure 7 A schematic diagram illustrating an application scenario for another traditional image correction method;

[0048] Figure 8 This is a diagram illustrating the application environment of an image correction method in one embodiment;

[0049] Figure 9 This is a structural block diagram of an image correction device in one embodiment;

[0050] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The image correction method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. The image correction method provided in this embodiment can also be applied only to terminal 102. In such an integrated embedded device, through AI algorithms, one or more of the processing, control, and storage involved in the AI ​​algorithm are incorporated into the solution of this application.

[0053] The terminal 102 can be, but is not limited to, various cameras, sensors, radars, personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0054] In one embodiment, such as Figure 2 As shown, an image correction method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0055] Step 202: Perform location symbol detection on the image to be processed to obtain the position of the location symbol in the image and the orientation represented by the location symbol;

[0056] In one embodiment, step 202 is a step performed after the terminal receives or detects a target event. The target event can be a preset event or an event that dynamically changes based on the user, controlling the terminal to perform location symbol detection on the acquired image and determine the next image processing step based on the current acquired image. Target events can be of various types, each with its own advantages. Setting corresponding activation strategies based on multiple target events simultaneously facilitates method execution. For example, when a target event is triggered by a specific gesture, the user can operate more conveniently; when a target event is triggered by a button, the information displayed by each button is clearer; when a target event is a terminal activation event, it can comprehensively record the image of the corresponding object; when a target event is a timed event, it can automate the corresponding process, especially suitable for classes or lectures; and combining multiple target events can combine the advantages of each type.

[0057] A locator symbol is a directional symbol used to define a region of interest obtained from a preset shape. A locator symbol is a component pre-positioned at a specific location; this component can be made of paper, plastic, metal, magnet, or any other material. Regardless of the material, the locator symbol's features indicate its orientation. For example, if the preset shape defined by the locator symbol is a rectangle, and the locator symbol represents one of the rectangle's corner points, then the locator symbol representing the top-left corner of the rectangle would be like this: Figure 3 Any one of (a)-(d) is shown.

[0058] In one embodiment, the positioning symbol is a "positioning sticker." This "positioning sticker" has certain characteristics that determine the indicated direction. By rotating it 90 degrees clockwise, positioning stickers at any given angle can be used to obtain positioning stickers indicating four different angles. The four positioning stickers are, in order, the top left, top right, bottom right, and bottom left corners. Figure 4 As shown.

[0059] Since the image is captured by a camera, it will be distorted due to acquisition parameters such as the image acquisition angle, resulting in a difference between the position of the positioning symbol in the image and its pre-placed actual position. Because of the directional indicative nature of the positioning symbol, detecting the positioning symbol in the image and determining the target area based on the detection results can achieve a high accuracy rate and a low false detection rate for target area recognition. The camera can be the terminal 102 itself, or various electronic devices with shooting capabilities, including gimbal cameras, video conferencing machines, action cameras, panoramic cameras, gimbal cameras, drones, etc.; this case does not limit the scope of the application.

[0060] The terminal can use the acquired image as the image to be processed in the location symbol detection process, or it can perform certain preprocessing on the acquired image and use the preprocessed image as the image to be processed. Preprocessing includes, but is not limited to, one or more of the following methods: image structure change processing, image smoothing, restoration processing, and image enhancement processing. Optionally, when performing multiple image corrections, the target image after the previous image correction can be used as the current image to be processed, until the corrected target image meets a certain specific condition.

[0061] In one embodiment, the detection of localization symbols in the image to be processed is performed using a template matching method. The template matching method identifies the region in the image to be processed that has the highest similarity to the localization symbol, thereby obtaining the localization symbol. For example, the template matching method includes: calculating the regions with similarity to the localization symbol in the image to be processed in a predetermined order, and then identifying the position of the localization symbol and its represented orientation based on the similarity of each region. Combining the template matching method with other methods can achieve better detection results. The methods combined with the template matching method can be one or more of threshold segmentation methods, color segmentation methods, and detection methods based on convolutional neural networks.

[0062] In one embodiment, the detection of localization symbols in an image is performed using a combination of threshold segmentation and template matching. Threshold segmentation divides the image into candidate regions, and then template matching is used within these candidate regions to find localization symbols, which speeds up the detection process. For example, the threshold segmentation method includes: calculating the image value of each unit region in the image; comparing the image value of each unit region with a corresponding threshold to determine whether the combined region formed by these unit regions is a candidate region; then processing the candidate regions using template matching to identify one or more localization symbols corresponding to the candidate regions, thereby obtaining the position of the localization symbol in the image and the orientation it represents.

[0063] In this context, a single unit region can be composed of one or more pixels, and a combined region can be composed of one or more unit regions. The corresponding threshold for image values ​​can be any type of value, such as grayscale or RGB values, or combinations thereof. When all pixel values ​​within a set threshold range are set to 1 and all others are set to 0, the image is binarized. When the corresponding threshold for image values ​​can be divided into multiple thresholds according to size, complex images with multiple objects can be processed, and the image can be divided into multiple different parts using multiple different thresholds.

[0064] In one embodiment, image localization symbol detection is performed using a combination of color segmentation and template matching. Color segmentation can be the same as threshold segmentation, or other methods may be used. For example, before color segmentation, the method includes: filtering the image to obtain a blurred image, retaining features with large differences and filtering out features with small differences in the blurred image; next, converting the image from BGR (blue, green, red) to HSV (hue and saturation values) to distinguish the values ​​of pixels B, G, and R from the light falling on the object, and reducing the coupling between these pixels to more accurately obtain the brightness, saturation, and chromaticity of the pixels. During color segmentation, the maximum and minimum values ​​of the localization symbols to be extracted are first determined, and regions in the image without localization symbols are removed based on the maximum and minimum values ​​to obtain candidate regions for localization symbols. Template matching is then performed on the candidate regions to identify the position of the localization symbol and the orientation it represents.

[0065] In one embodiment, performing location symbol detection on the image to be processed to obtain the position of the location symbol in the image and the orientation represented by the location symbol includes: in response to a target event, obtaining a detection model of the location symbol; detecting the location symbol in the image according to the detection model to obtain the position of the location symbol in the image; and obtaining the orientation represented by the location symbol based on the features of the location symbol.

[0066] The detection model for location symbols is a pre-trained detection model specifically for location symbols. Detection models can include YOLO (You Only Look Once), SSD (Single Shot MultiBox), R-CNN (Region-based Convolutional Neural Networks), or Mask R-CNN (Mask Region-based Convolutional Neural Networks), etc. These models can detect location symbols, determine their position in the image, and obtain the orientation represented by the location symbols based on their features.

[0067] The features of a positioning symbol are its structural features, which can be set according to one or more feature dimensions such as color, grayscale value, and positioning symbol shape. When the feature dimensions include predefined positioning symbol shape and color, combined with an advanced target detection model, it can detect various changes in the target, exhibiting high robustness, superior stability, and high performance. Moreover, it is easy to obtain accurate positioning results, effectively avoiding false detections of similar objects while ensuring the detection rate, thus achieving accurate identification of positioning symbols and accurately determining the region of interest in the image.

[0068] The regions obtained by connecting the locations of the positioning symbols differ from the preset shapes. This is due to image distortion caused by factors such as image acquisition angle and alternation coefficients. The degree of distortion varies greatly, making it necessary to filter the regions determined by the positioning symbols. The filtering steps include: detecting based on the number of positioning symbols to determine if the number of positioning symbols meets the preset conditions; and matching the positions of the positioning symbols with their orientations to determine if the positions and orientations correspond.

[0069] Step 204: When the number of location symbols detected meets the preset conditions and the positions correspond to the orientations, the location symbols are connected to obtain the target area of ​​the location symbols.

[0070] When the number of location symbols detected meets a preset condition, the image is determined to be correctable based on the number of location symbols. This means the target area of ​​the location symbols can be resolved, and the difference between the target area and the preset shape is within a certain range. For example, if there are four location symbols and the preset shape is a rectangle, then the number of location symbols meets the preset condition; if there are three location symbols and the preset shape is a triangle, then the number of location symbols meets the preset condition.

[0071] Determining the correspondence between location and orientation involves judging whether an image can be corrected based on the position of the location symbols. An image is considered abnormal if the orientation represented by a location symbol differs from its actual location. For example, if the preset shape is a rectangle, and the location symbols for the rectangle represent the top left, top right, bottom right, and bottom left corners, then an image can be considered correctable if the location symbols in the image correspond to these four corners one-to-one. However, if the user places an incorrect location symbol, such as placing the top-left symbol in the bottom right corner of the image, an abnormality occurs, and no valid rectangle is detected.

[0072] In one embodiment, during the process of connecting the positioning symbols, the terminal uses the positioning symbols as the edge corners of the region of interest, and uses the edge corners as the vertices of the region of interest to obtain the target region. For example, each vertex of the rectangle can be used as the orientation represented by the positioning symbol to determine the location of the region of interest.

[0073] In one embodiment, connecting the positioning symbols to obtain the target area of ​​the positioning symbols includes: connecting the positioning symbols to obtain an initial area; determining whether the boundary of the initial area matches the boundary of a preset shape; if not, filtering out the initial area; if so, filtering the initial area according to the boundary angle to determine whether the initial area is the target area.

[0074] The initial region is the region obtained after preliminary screening. By detecting the number of boundaries of the initial region according to the preset shape, the initial regions with excessive deformation can be filtered out. Then, the initial regions with less distortion can be filtered again according to the boundary angle, and the region obtained after the second filtering is determined as the target region.

[0075] A method for determining whether the boundary of the initial region matches the boundary of the preset shape can be based on whether the number of boundaries matches, so as to quickly and accurately filter out the initial region with excessive deformation. In one embodiment, when the number of boundaries in the region exceeds the number of boundaries of the preset shape, the terminal may incorrectly detect some symbols similar to the positioning symbols as positioning symbols, resulting in misidentified positioning symbols; when the number of boundaries in the region is less than the number of boundaries of the preset shape, a positioning symbol is not connected to an adjacent positioning symbol.

[0076] The method for determining whether boundaries match can also be based on region area filtering. This involves comparing the area of ​​the initial region with the area enclosed by the boundary of a preset shape, where the area can be pre-defined. The area of ​​the initial region can be calculated using a region growing algorithm or other area calculation methods. A region growing algorithm uses a location symbol or arbitrary coordinate point as a seed, selects neighboring pixels with similar properties as the seed's growth region, and uses the edge points of this growth region as the seed for the next region growing iteration. When the next seed for region growing is located at the boundary of the initial region, the area of ​​the initial region is calculated. Neighboring pixels with similar properties to the seed are defined as those whose grayscale or color dimension differences are less than a certain range relative to the seed.

[0077] Methods for determining whether boundaries match can be based on region shape filtering. This can be based on matching the boundaries of the initial region with the boundaries of a preset shape, or it can use the difference in gradient vectors of pixel features as a similarity criterion between two images, based on whether the boundaries of the initial region match the boundaries of the preset shape.

[0078] In one embodiment, filtering the initial region based on the boundary angle to determine whether the initial region is a target region includes: calculating the boundary angle of the initial region; filtering the initial region according to whether the boundary angle is within the angle range of a preset shape; and using the filtered initial region as the target region.

[0079] The boundary angle of the initial region is the angle between the adjacent boundaries of the initial region. When the boundary angle is within the range of the preset shape, the distortion of the image is small and can be corrected; when the boundary angle is outside the range, the distortion of the image is large and difficult to correct, and the initial region is not used as the target region.

[0080] In one embodiment, the preset shape is a rectangle, and the positioning symbols are the corner points of the rectangle. During the filtering process of the four detected positioning symbols according to the rectangle, if the location of the user-installed positioning sticker exactly forms the four vertices of the rectangle, the initial region in the image has minimal distortion, and this initial region can be used as the target region. However, if the region defined by the location of the user-installed positioning sticker is a very irregular quadrilateral, it is difficult to guarantee a good correction effect after projecting the target region. Based on this design algorithm, very irregular quadrilaterals are filtered out.

[0081] Correspondingly, a feasible filtering method is to connect the four detected location symbols in pairs in a clockwise order (i.e., connect the top left + top right, top right + bottom right, bottom right + bottom left, and bottom left + top left) to obtain a quadrilateral. Then, calculate the included angles of each boundary of the quadrilateral. When the included angle of each boundary meets the range of "greater than or equal to 60 degrees and less than or equal to 120 degrees", the initial area is a valid rectangle, and the detection is considered successful, and the initial area is determined as the target area. Otherwise, the detection is considered a failure, and feedback is provided. The feedback can be provided by increasing the detection time and continuing the detection, or by issuing certain information to remind the user that the location symbols are pasted incorrectly.

[0082] In one embodiment, the positioning symbols include a first positioning symbol and a second positioning symbol. The first positioning symbol is a detected positioning symbol, and the aforementioned position and orientation refer to the position and orientation of the first positioning symbol. Correspondingly, the method further includes: when the number of the first positioning symbols does not meet a preset condition, determining a detection area for the second positioning symbol based on the orientation; detecting the second positioning symbol within the detection area to obtain the updated number, updated position, and updated orientation of the first positioning symbol; when the updated number meets the preset condition, and the updated position and updated orientation are consistent, connecting the updated positioning symbols to obtain the target area of ​​the positioning symbol. Thus, by determining the detection area based on the orientation represented by the detected positioning symbols, searching for undetected positioning symbols within that detection area, and then determining the required detection area, the object being photographed is changed to acquire an image in the target area of ​​the positioning symbol.

[0083] In one embodiment, determining the detection area of ​​the second positioning symbol based on the orientation represented by the first positioning symbol includes: adjusting the acquisition parameters of the imaging device according to the orientation represented by the first positioning symbol; re-acquiring an image based on the adjusted acquisition parameters; and determining the detection area of ​​the second positioning symbol based on the orientation in the re-acquired image.

[0084] The acquisition parameters of the shooting device are adjusted. The acquisition parameters can be the acquisition direction or the acquisition direction and focal length. The first image is the currently acquired image, the first positioning symbol is the positioning symbol detected in the first image, and the second positioning symbol is the positioning symbol that is not detected.

[0085] In one embodiment, the first and second positioning symbols in each image are described at different stages of the image correction method implementation. Before the positioning symbol detection stage of the image to be processed, the first positioning symbol does not exist in the image to be processed. During the positioning symbol detection stage of the image to be processed, if the number of positioning symbols in the image does not meet a preset condition, the positioning symbols in the image are divided into the first and second positioning symbols. The acquisition parameters of the imaging device are adjusted according to the orientation represented by the first positioning symbol to re-acquire the image. Before the stage of using the re-acquired image as the image to be processed in the image correction process, the first positioning symbol does not exist in the re-acquired image. Therefore, determining the detection area of ​​the second positioning symbol according to the orientation is actually determining the detection area of ​​each positioning symbol in the re-acquired image. The re-acquired image can be used as the image to be processed in the next execution process of the image correction method until the execution process of the image correction method is completed or terminated.

[0086] In one embodiment, after adjusting at least one of the acquisition direction and focal length of the shooting device, the second positioning symbols in the re-acquired image are all able to identify their corresponding positions and the orientations they represent during the positioning symbol detection process, so as to obtain the target area of ​​the positioning symbol.

[0087] The orientation represented by the first positioning symbol is determined based on at least one first positioning symbol. When the orientation represented by a positioning symbol is upper left, the acquisition direction will be adjusted to lower right, and the focal length can also be adjusted. When the orientation represented by a first positioning symbol is upper left, and the orientation represented by another first positioning symbol is lower left, the acquisition direction is determined based on these two orientations and adjusted horizontally to the right. After adjusting the acquisition parameters, the image is re-acquired based on the adjusted acquisition parameters, and the detection area of ​​the second symbol is determined in the re-acquired image. In this way, the region of interest can be automatically found even when only part of the region of interest is in the frame, with high accuracy, high intelligence, and a good interactive experience.

[0088] Step 206: Correct the target area to obtain the corrected target image.

[0089] The target image is the image obtained after correcting the target region. This image maps the target region onto another plane to correct perspective distortion. The correction achieved through perspective transformation in this process can be one or more of rigid transformation, similarity transformation, and affine transformation. Specific correction methods can be one or more of digital correction and optical correction.

[0090] In one embodiment, the above correction is a quadrilateral correction process. Quadrilateral correction can be digital correction, which works by applying a geometric transformation to the original image using an interpolation algorithm to address rectangular distortion caused by the imaging optical path, so that the rectangular target area appears as a regular rectangle in the processed output image. Quadrilateral correction can also be optical correction. Optical correction can be achieved by adjusting the physical position of the lens to adjust the shape of quadrilaterals in the image.

[0091] In one embodiment, correcting a target region to obtain a corrected target image includes: generating a corresponding perspective transformation matrix according to the vertex positions of a preset shape; and correcting the target region according to the perspective transformation matrix to obtain the corrected target image.

[0092] The vertex positions of the preset shape are multiple non-collinear positions that are pre-defined. These positions are calculated according to projection to obtain a perspective transformation matrix. Then, the positions of each positioning symbol in the target area are calculated according to the perspective transformation matrix to correct the target area and obtain the corrected target image.

[0093] The method also includes: displaying the target image.

[0094] The target image can be the first target image of the rectified target region, or a second target image obtained by performing certain display enhancement processing on the first target image. One method of display enhancement processing is to sharpen the target image and improve its clarity by blurring the boundaries and increasing the sharpness of the region of interest and the focal length of the image. Another method is to increase the difference in grayscale values ​​of the region of interest to enhance the contrast effect of the target image and improve its clarity. The second target image can also be obtained by magnification or other processing methods.

[0095] In the above image correction method, the image is subjected to location symbol detection to obtain the position of the location symbol in the image and the orientation represented by the location symbol. When the number of detected location symbols meets the preset conditions and the position corresponds to the orientation, the location symbols are connected to obtain the target area of ​​the location symbol. Thus, the region of interest is determined based on the location symbol, without the need to identify the region of interest through edge detection algorithm, reducing the possibility of misidentification and achieving high accuracy. Furthermore, the target image obtained by correcting the target area achieves image correction. After obtaining the corrected target image, other display enhancement processing methods can be used to further enhance, enlarge, or process the target image, making the displayed image clearer or able to meet other user needs.

[0096] In one embodiment, the application scenario of this solution is described. In application scenarios such as video conferencing, live online courses, and recorded courses, one type of user requirement is to ensure the correction effect of the whiteboard content during display / recording. Generally speaking, the correction effect includes the following requirements: 1. Since the camera usually cannot be directly facing the whiteboard, the content on the whiteboard image is distorted to a certain extent. Users need to correct the whiteboard area to a rectangle for easier reading; 2. There is a need for visual enhancement, such as wanting to enlarge the whiteboard area to make details clearer. Regardless of which requirement is met, the prerequisite is to find the positions of the four corner points of the whiteboard in the image (automatically or manually).

[0097] "Whiteboard" is a broad term referring to any flat surface that can be used to correct information, including but not limited to whiteboards, semi-transparent whiteboards, blackboards, white walls, electronic whiteboards, wallpaper, glass surfaces that can be written on, television screens, monitors, projection screens, and electronic whiteboards.

[0098] Compared to a traditional solution in the industry, the solution in this application not only eliminates the need for a dedicated camera on the whiteboard, but also allows for the capture of various objects, including borderless whiteboards, with high accuracy. The traditional solution involves installing a dedicated camera above the whiteboard, detecting the whiteboard in the captured image, and determining the positions of its four corner points (which can be manually corrected in the included software if the user deems inaccurate). A corrected rectangular image is then generated based on these corner points. However, this method requires the camera to be installed beforehand, which is inconvenient; the camera can only capture the whiteboard image and cannot capture the speaker or other objects; more importantly, this method can only probabilistically identify whiteboards with boundaries, failing to recognize borderless or poorly defined whiteboards, and its accuracy is usually insufficient. After installing the camera and whiteboard and performing automatic recognition, users typically need to manually adjust the recognition results on the client-side. Its application scenarios include... Figure 5 As shown.

[0099] Compared to another traditional solution in the industry, this solution has higher accuracy and is less prone to missed detections or misinterpretations. Another traditional solution involves the user fixing the camera to a table, chair, or stand in front of a whiteboard, then using an algorithm to detect the whiteboard and perform image correction and enhancement based on the positions of the four corner points of the whiteboard. This method is prone to misdetecting some white areas or closed rectangular areas as whiteboards, and is also prone to missed detections due to unclear edges of the whiteboard, resulting in low accuracy and poor usability. This solution has two specific implementation methods: one is based on edge detection, detecting line segments in the image and then forming rectangles; the other is based on color threshold segmentation and then fitting rectangular regions. The application scenarios for these two implementation methods are as follows: Figure 6 , Figure 7 As shown.

[0100] In one embodiment, the overall process of this solution is illustrated through an application scenario.

[0101] First, after reading the instructions, the user understands that they need to install the positioning stickers onto the four corner points of the area of ​​interest, following the directions indicated, and then turn on the camera to record. The area of ​​interest is the user's whiteboard, and the whiteboard and positioning stickers are as follows: Figure 8 As shown.

[0102] Next, when responding to a target event, after initiating whiteboard mode, the terminal automatically detects the positioning symbol in the captured image. The method for detecting the positioning symbol is to use a detector based on a convolutional neural network to calculate the position of the positioning symbol in the image and the orientation it represents.

[0103] When four positioning symbols are detected at once, and the positions of the four positioning symbols correspond to the directions they represent, the location of the whiteboard is determined as the target area. Image acquisition can be performed by adjusting the image acquisition direction according to the target area, or projection mapping can be performed directly.

[0104] If fewer than four location symbols are detected, the detected symbols are designated as the first location symbols, and the undetected symbols are designated as the second location symbols. The detection area for the second location symbols is determined based on the position and orientation of the first location symbols, aiming to convert as many second location symbols as possible into first location symbols, thus obtaining the updated number, position, and orientation of the first location symbols. When the updated number is four, and the updated position and orientation are consistent, the updated location symbols are connected to obtain the target area for the location symbols. The detection area is checked every 0.5 seconds; if no success is found after 10 consecutive seconds of checks, the system exits the whiteboard mode.

[0105] After detecting the target area, the projected image obtained by projecting the target area can be used to correct this quadrilateral area. It can also perform image magnification, sharpening, and contrast enhancement to improve image clarity.

[0106] To more accurately correct the image, a step of filtering the target area according to rectangles is included before correction. It's important to understand that if the four positioning stickers applied by the user form a perfect rectangle, or the error is very small, the corrected visual effect will be better. If the shape deviates significantly from the standard rectangle, the final display effect may be poor. We can design an algorithm to filter such irregular quadrilaterals. The designed filtering algorithm is as follows: Connect the four detected positioning symbols in pairs in a clockwise order (i.e., connect the top left + top right, top right + bottom right, bottom right + bottom left, bottom left + top left) to determine the quadrilateral region; then, judge the included angle of each boundary of the quadrilateral region. If each included angle satisfies "greater than or equal to 60 degrees and less than or equal to 120 degrees", it is considered a valid rectangle, and the detection is successful, proceeding to the next step; otherwise, the detection is considered a failure, and the detection continues.

[0107] Therefore, this embodiment proposes a whiteboard detection and correction scheme based on positioning stickers. The user affixes positioning stickers to the four corner points of the region of interest (ROI) on the whiteboard, which is to be corrected, according to a prescribed pattern. The terminal automatically identifies these four corner points using image algorithms and performs rectangular correction, magnification, and enhanced display clarity on the ROI. This embodiment does not require the ROI to have obvious boundaries and boasts advantages such as extremely high recognition accuracy and extremely low false detection rate. Furthermore, for cameras with gimbals, this embodiment's scheme can automatically locate the whiteboard even when only part of it is in the frame, demonstrating high intelligence and a good user experience.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this application also provides an image correction apparatus for implementing the image correction method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image correction apparatus embodiments provided below can be found in the limitations of the image correction method described above, and will not be repeated here.

[0110] In one embodiment, such as Figure 9 As shown, an image correction device is provided, including: a detection module 902, a target region determination module 904, and an image correction module 906, wherein:

[0111] The detection module 902 is used to detect positioning symbols in the image to be processed, and to obtain the position of the positioning symbol in the image and the orientation represented by the positioning symbol.

[0112] The target area determination module 904 is used to connect the positioning symbols to obtain the target area of ​​the positioning symbols when the number of the detected positioning symbols meets the preset conditions and the position corresponds to the orientation.

[0113] The image correction module 906 is used to correct the target area to obtain the corrected target image.

[0114] In one embodiment, the detection module 902 is used for:

[0115] A detection model for acquiring location symbols in response to target events;

[0116] The location symbols in the image are detected according to the detection model to obtain the position of the location symbols in the image and the features of the location symbols;

[0117] Based on the characteristics of the positioning symbol, the orientation represented by the positioning symbol is obtained.

[0118] In one embodiment, the target region determination module 904 is used to:

[0119] Connect the positioning symbols to obtain the initial area;

[0120] Determine whether the boundary of the initial region matches the boundary of the preset shape;

[0121] If not, then the initial region is filtered out;

[0122] If so, the initial region is filtered based on the boundary angle to determine whether the initial region is the target region.

[0123] In one embodiment, the target region determination module 904 is specifically used for:

[0124] Calculate the included angle of the boundary of the initial region;

[0125] The initial region is filtered according to whether the included angle of the boundary is within the included angle range of the preset shape;

[0126] The filtered initial region is used as the target region.

[0127] In one embodiment, the positioning symbol includes a first positioning symbol and a second positioning symbol, wherein the first positioning symbol is a detected positioning symbol, the second positioning symbol is an undetected positioning symbol, and the position and orientation are the position and orientation of the first positioning symbol.

[0128] Correspondingly, the target area determination module 904 is also used to determine the detection area of ​​the second positioning symbol based on the orientation when the number of the first positioning symbols does not meet the preset conditions;

[0129] The detection module 902 is further configured to detect the second positioning symbol in the detection area to obtain the updated number, updated position and updated orientation of the first positioning symbol;

[0130] The target area determination module 904 is further configured to connect the updated positioning symbols to obtain the target area of ​​the positioning symbols when the updated quantity meets the preset condition and the updated position is consistent with the updated orientation.

[0131] In one embodiment, the detection module 902 is further configured to: adjust the acquisition direction of the image according to the orientation represented by the first positioning symbol; and re-acquire the image according to the adjusted acquisition direction.

[0132] Correspondingly, the target area determination module 904 is also used to determine the detection area of ​​the second positioning symbol in the re-acquired image based on the orientation.

[0133] In one embodiment, the image correction module 906 is used for:

[0134] Generate the corresponding perspective transformation matrix according to the vertex positions of the preset shape;

[0135] The target region is projected and corrected according to the perspective transformation matrix to obtain a projected image of the corrected target region.

[0136] By executing the methods of any of the above embodiments using this device, the accuracy of region of interest identification can be improved.

[0137] Each module in the aforementioned image correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0138] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image correction method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0139] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0144] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0145] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image correction method characterized by, The method includes: The image to be processed is subjected to location symbol detection to obtain the position of the location symbol in the image and the orientation represented by the location symbol; the orientation is the orientation on the preset shape represented by the location symbol. When the number of the positioning symbols meets the preset condition and the position corresponds to the orientation, the positioning symbols are connected to obtain the target area of ​​the positioning symbols; the target area is the area corresponding to the preset shape. The target region is corrected to obtain the corrected target image.

2. The method according to claim 1, characterized in that, The step of performing location symbol detection on the image to be processed to obtain the position of the location symbol in the image and the orientation represented by the location symbol includes: A detection model for acquiring location symbols in response to target events; The location symbols in the image are detected according to the detection model to obtain the position of the location symbols in the image and the features of the location symbols; Based on the characteristics of the positioning symbol, the orientation represented by the positioning symbol is obtained.

3. The method according to claim 1, characterized in that, The step of connecting the positioning symbols to obtain the target area of ​​the positioning symbols includes: Connect the positioning symbols to obtain the initial area; Determine whether the boundary of the initial region matches the boundary of the preset shape; If not, then the initial region is filtered out; If so, the initial region is filtered based on the boundary angle to determine whether the initial region is the target region.

4. The method according to claim 3, characterized in that, The step of filtering the initial region based on the boundary angle to determine whether the initial region is the target region includes: Calculate the included angle of the boundary of the initial region; The initial region is filtered according to whether the included angle of the boundary is within the included angle range of the preset shape; The filtered initial region is used as the target region.

5. The method according to claim 1, characterized in that, The positioning symbol includes a first positioning symbol and a second positioning symbol, wherein the first positioning symbol is a detected positioning symbol, the second positioning symbol is an undetected positioning symbol, and the position and orientation are the position and orientation of the first positioning symbol. The method further includes: When the number of the first positioning symbols does not meet the preset condition, the detection area of ​​the second positioning symbol is determined according to the orientation; In the detection area, the second positioning symbol is detected to obtain the updated number, updated position, and updated orientation of the first positioning symbol; When the updated quantity meets the preset condition and the updated position is consistent with the updated orientation, the updated positioning symbols are connected to obtain the target area of ​​the positioning symbols.

6. The method according to claim 5, characterized in that, The step of determining the detection area of ​​the second positioning symbol based on the orientation represented by the first positioning symbol includes: Adjust the acquisition parameters of the shooting device according to the orientation represented by the first positioning symbol; Based on the adjusted acquisition parameters, reacquire the image; In the re-acquired image, the detection area of ​​the second positioning symbol is determined based on the orientation.

7. The method according to claim 1, characterized in that, The step of correcting the target region to obtain the corrected target image includes: Generate the corresponding perspective transformation matrix according to the vertex positions of the preset shape; The target region is corrected according to the perspective transformation matrix to obtain the corrected target image.

8. An image correction device, characterized in that, The device includes: The detection module performs positioning symbol detection on the image to be processed, and obtains the position of the positioning symbol in the image and the orientation represented by the positioning symbol; the orientation is the orientation on the preset shape represented by the positioning symbol; The target area determination module is used to connect the positioning symbols when the number of detected positioning symbols meets a preset condition and the position corresponds to the orientation, to obtain the target area of ​​the positioning symbols; the target area is the area corresponding to the preset shape. The image correction module is used to correct the target area to obtain the corrected target image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images