Image conversion method and terminal device

Abstract

The embodiment of the present invention discloses an image conversion method and a terminal device, which are applied to the field of terminal device technology and can convert a deformed image into a standard image to enhance recognition accuracy. The method comprises: collecting a first image including the light and shadow; calculating the RGB value of each pixel in the light and shadow, and determining the pixel with the largest RGB value as the center of gravity of the brightness value of the light and shadow; determining the optical axis according to the light and shadow projector and the center of gravity of the brightness value; projecting the first image onto a plane perpendicular to the optical axis through perspective transformation technology to obtain a target image. The method is applied to a scene in which a terminal device performs photo recognition.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of terminal devices, and in particular to an image conversion method and a terminal device. Background Art

[0002] With the development of science and technology, smart watches are becoming more and more popular and their functions are becoming more and more powerful. At present, smart watches can also be used to query information. When children encounter knowledge points that they don’t know during the learning process, they can use the camera to collect image information of the book and use optical character recognition (OCR) technology to obtain the content they need to query.

[0003] However, when children use a camera to take pictures of books, it is difficult to ensure a standard bird's-eye view. Generally, there will be a certain deviation angle, which will cause the collected image to not be a standard rectangle. It will undergo certain deformations, such as becoming a trapezoid, and the text will also be deformed, affecting the recognition accuracy. Summary of the invention

[0004] The embodiment of the present invention provides an image conversion method and a terminal device to solve the problem in the prior art that the collected images and texts will be deformed to a certain extent, affecting the recognition accuracy. In order to solve the above technical problems, the embodiment of the present invention is implemented as follows:

[0005] In a first aspect, an image conversion method is provided, the method comprising: acquiring a first image including the light and shadow;

[0006] Calculate the RGB value of each pixel in the light and shadow, and determine the pixel with the largest RGB value as the center of gravity of the brightness value of the light and shadow;

[0007] Determine the optical axis according to the light and shadow projector and the center of gravity of the brightness value;

[0008] The first image is projected onto a plane vertical to the optical axis by using a perspective transformation technique to obtain a target image.

[0009] As an optional implementation manner, in a first aspect of the embodiments of the present invention, the collecting the first image including the light and shadow includes:

[0010] capturing the first image and a second image, wherein the second image does not include the light and shadow;

[0011] Obtaining the RGB value of each pixel in the first image and the RGB value of each pixel in the second image;

[0012] A difference analysis is performed on the RGB values ​​of each corresponding pixel in the first image and the second image, and the light and shadow are determined according to the pixels in the first image whose RGB value differences are greater than a preset value.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, projecting the first image onto a plane vertical to the optical axis by using a perspective transformation technology to obtain a target image includes:

[0014] Determining a vertical plane of the optical axis through a target pixel point, wherein the target pixel point is any pixel point in the first image;

[0015] Projecting all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points;

[0016] Determine a third image according to the projected pixel points and the target pixel points;

[0017] All pixels in the third image are smoothed to obtain the target image.

[0018] As an optional implementation manner, in the first aspect of the embodiment of the present invention, projecting all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points includes:

[0019] Establishing a circumscribed rectangle of the light and shadow, and obtaining N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4;

[0020] Establishing a first coordinate axis through the target pixel point, obtaining first coordinate values ​​of the N tangent points, wherein the target pixel point is specifically any one of the N tangent points;

[0021] Substituting the first coordinate values ​​of the N tangent points into the perspective change expression to calculate a first perspective transformation matrix;

[0022] The first coordinate values ​​of all pixels in the first image are multiplied by the first perspective transformation matrix to obtain second coordinate values ​​of all pixels.

[0023] As an optional implementation manner, in the first aspect of the embodiment of the present invention, when there is a visualized straight line in the light and shadow, after acquiring the first image including the light and shadow, the method further includes:

[0024] Acquire the coordinate value of the first endpoint of the visualized straight line, the coordinate value of the second endpoint of the visualized straight line, and the coordinate value of the midpoint between the first endpoint and the second endpoint;

[0025] Calculate a first slope of a straight line between the first endpoint and the midpoint and a second slope of a straight line between the second endpoint and the midpoint;

[0026] calculating a difference between the first slope and the second slope;

[0027] If the difference is greater than a preset difference, a prompt message is output, where the prompt message is used to prompt the user that the learning page to be photographed is not flat.

[0028] In a second aspect, a terminal device is provided, the terminal device comprising: an acquisition module, configured to acquire a first image including the light and shadow;

[0029] A processing module, used to calculate the RGB value of each pixel in the light and shadow;

[0030] A determination module, used to determine the pixel point with the largest RGB value as the center of gravity of the brightness value of the light and shadow;

[0031] The determination module is further used to determine the optical axis according to the light and shadow projector and the center of gravity of the brightness value;

[0032] The processing module is further used to project the first image onto a plane vertical to the optical axis through a perspective transformation technique to obtain a target image.

[0033] As an optional implementation, in the second aspect of the embodiment of the present invention, the acquisition module is further used to collect the first image and the second image, and the second image does not include the light and shadow;

[0034] The acquisition module is further used to acquire the RGB value of each pixel in the first image and the RGB value of each pixel in the second image;

[0035] The processing module is further used to perform difference analysis on the RGB value of each corresponding pixel point in the first image and the second image;

[0036] The determination module is further used to determine the light and shadow according to the pixel points in the first image whose RGB value differences are greater than a preset value.

[0037] As an optional implementation, in the second aspect of the embodiment of the present invention, the determination module is further used to determine the vertical plane of the optical axis through a target pixel point, wherein the target pixel point is any pixel point in the first image;

[0038] The processing module is further used to project all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points;

[0039] The determination module is further used to determine a third image according to the projected pixel points and the target pixel points;

[0040] The processing module is further used to perform smoothing processing on all pixels in the third image to obtain the target image.

[0041] As an optional implementation, in the second aspect of the embodiment of the present invention, the processing module is further used to establish a circumscribed rectangle of the light and shadow, and obtain N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4;

[0042] The processing module is further used to establish a first coordinate axis through the target pixel point to obtain first coordinate values ​​of the N tangent points, wherein the target pixel point is specifically any one of the N tangent points;

[0043] The processing module is further used to substitute the first coordinate values ​​of the N tangent points into the perspective change expression to calculate a first perspective transformation matrix;

[0044] The processing module is further used to multiply the first coordinate values ​​of all pixels in the first image by the first perspective transformation matrix to obtain second coordinate values ​​of all pixels.

[0045] As an optional implementation, in the second aspect of the embodiment of the present invention, when there is a visualized straight line in the light and shadow,

[0046] The acquisition module is further used to acquire the coordinate value of the first endpoint of the visualized straight line, the coordinate value of the second endpoint of the visualized straight line, and the coordinate value of the midpoint between the first endpoint and the second endpoint;

[0047] The processing module is further used to calculate a first slope of a straight line where the first endpoint and the midpoint are located and a second slope of a straight line where the second endpoint and the midpoint are located;

[0048] The processing module is further used to calculate the difference between the first slope and the second slope;

[0049] The terminal device further includes:

[0050] The output module is used to output a prompt message if the difference is greater than a preset difference, wherein the prompt message is used to prompt the user that the learning page to be photographed is not placed flat.

[0051] In a third aspect, a terminal device is provided, including:

[0052] A memory storing executable program code;

[0053] a processor coupled to the memory;

[0054] The processor calls the executable program code stored in the memory to execute the image conversion method in the first aspect of the embodiment of the present invention.

[0055] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, wherein the computer program enables a computer to execute the image conversion method in the first aspect of the embodiment of the present invention. The computer-readable storage medium includes a ROM / RAM, a magnetic disk or an optical disk, etc.

[0056] According to a fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute part or all of the steps of any one of the methods according to the first aspect.

[0057] According to a sixth aspect, an application publishing platform is provided, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of any one method of the first aspect.

[0058] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0059] In an embodiment of the present invention, the terminal device can project light and shadow on the learning page to be photographed and collect images including the light and shadow, and determine the pixel point with the largest brightness value in the light and shadow as the center of gravity of the brightness value. At this time, the optical axis can be obtained. Because the camera of the terminal device and the learning page to be photographed are not parallel, there is a certain angle between the optical axis and the collected image, so that the image will be deformed. Finally, the collected image is projected onto the vertical plane of the optical axis through perspective transformation technology. At this time, the image and the optical axis can be guaranteed to be completely vertical, and the image without deformation after correction can be obtained. In this way, not only can the content required by the user be identified in a targeted manner, but also the accuracy of image recognition and text recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1a 1 is a flow chart of an image conversion method provided by an embodiment of the present invention;

[0062] Figure 1b is a scene schematic diagram of an image conversion method provided by an embodiment of the present invention;

[0063] Figure 2 The following is a schematic diagram of a process of an image conversion method provided by an embodiment of the present invention. Figure 2 ;

[0064] Figure 3 The following is a schematic diagram of a process of an image conversion method provided by an embodiment of the present invention. Figure 3 ;

[0065] Figure 4 1 is a structural schematic diagram of a terminal device provided by an embodiment of the present invention;

[0066] Figure 5 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention. Figure 2 ;

[0067] Figure 6 It is a schematic diagram of the hardware structure of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The terms "first" and "second" in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first image and a second image are used to distinguish different images rather than to describe a specific order of the images.

[0070] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatus.

[0071] It should be noted that, in the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0072] The embodiment of the present invention provides an image conversion method and a terminal device, which can convert a deformed image into a standard image and enhance recognition accuracy.

[0073] The terminal device involved in the embodiment of the present invention may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA) and other electronic devices. Among them, the wearable device may be a smart watch, a smart bracelet, a watch phone, a smart anklet, a smart earring, a smart necklace, a smart headset, etc., which is not limited in the embodiment of the present invention.

[0074] The execution subject of the image conversion method provided by the embodiment of the present invention can be the above-mentioned terminal device, or it can be a functional module and / or functional entity in the terminal device that can implement the image conversion method. The specific can be determined according to actual use requirements, and the embodiment of the present invention does not limit it. The following takes the terminal device as an example to exemplarily illustrate the image conversion method provided by the embodiment of the present invention.

[0075] The image conversion method provided by the embodiment of the present invention can be applied to the scenario where a terminal device performs photo recognition.

[0076] Embodiment 1

[0077] like Figure 1a As shown, an embodiment of the present invention provides an image conversion method, which may include the following steps:

[0078] 101. Collect a first image including light and shadow.

[0079] In the embodiment of the present invention, a fixed light and shadow projector is provided in the camera of the terminal device, and the light and shadow projector can project light and shadow within the acquisition range of the camera, and the light and shadow will move as the angle of the camera changes.

[0080] Optionally, the light and shadow can be set to a variety of shapes. In order to facilitate calculation, the shape of the light and shadow is generally a regular symmetrical shape, such as: circle, square, cross, etc.

[0081] When the user needs to take a photo to search for some content during the learning process, the user can place the learning page to be photographed within the shooting range of the camera of the terminal device and issue a first command. The terminal device responds to the user's first command, turns on the camera and projects light and shadow onto the learning page to be photographed through a light and shadow projector. The learning page to be photographed can be a textbook or exercise book.

[0082] It should be noted that the first instruction can be in various forms, such as a user clicking a virtual control or a physical button in the terminal device, or a gesture instruction set by the user, or a voice instruction input by the user for activation, which is not limited in the embodiment of the present invention.

[0083] Optionally, the first instruction issued by the user can be implemented in three ways:

[0084] Implementation method 1: The first instruction sent by the user is used to start the camera of the terminal device to collect images, and automatically start the light and shadow projector in the camera to project light and shadow onto the learning page to be photographed.

[0085] Implementation method 2: The first command issued by the user is only used to start the camera of the terminal device to capture images. At this time, the user needs to issue a second command again to start the light projector in the camera to project light and shadow onto the learning page to be photographed.

[0086] Implementation method three: The first command issued by the user is only used to start the camera of the terminal device to capture images. After capturing an image, the terminal device detects whether the image and the content therein are deformed. If the terminal device detects that the image and the content therein are not a standard rectangle and are deformed, the terminal device can control the light and shadow projector in the camera to project light and shadow onto the learning page to be photographed.

[0087] After the terminal device projects the light and shadow on the learning page to be photographed, the user can adjust the position of the camera according to the position of the light and shadow projection, and move the light and shadow to the specific learning page to be photographed. The learning page to be photographed can be a specific formula, an exercise or a word, etc. The terminal device controls the camera to capture the first image including the light and shadow.

[0088] Optionally, when the terminal device collects the first image including light and shadow, it can also adjust the shape of the light and shadow according to the content in the learning page to be photographed. When the content that the user wants to search is a small range of content, such as a word or an idiom, the terminal device can respond to the user's third instruction to control the light and shadow projector in the camera to project a circular or cross-shaped light and shadow; when the content that the user wants to search is a large range of content, such as an ancient poem or a paragraph, the terminal device can respond to the user's fourth instruction to control the light and shadow projector in the camera to project a rectangular light and shadow that can be adjusted in size. Among them, the third instruction and the fourth instruction can be a user's click operation on a virtual control or physical button in the terminal device, or a gesture instruction set by the user, or a voice instruction input by the user for opening, which is not limited in the embodiment of the present invention.

[0089] This optional implementation method can select different light and shadow shapes according to different content to be photographed, so as to avoid the light and shadow being too large to cover additional content, and to avoid the light and shadow being too small to cover all content, thereby reducing the probability of errors in image recognition.

[0090] Optionally, the shooting angle of the camera in the terminal device can be adjusted. Before collecting the first image including light and shadow, the camera angle can also be adjusted. When it is detected that the learning page to be photographed is within the shooting range of the camera, the camera is fixed and the first image including light and shadow is collected. If the learning page to be photographed is not detected to be within the shooting range of the camera, the shooting angle of the camera is adjusted; if the learning page to be photographed is still not detected to be within the shooting range of the camera within a preset time, a prompt message is output to the user, which is used to prompt the user that the learning page has not been photographed, or to suggest the user to adjust the camera angle or adjust the position of the learning page to be photographed.

[0091] The above optional implementation method can output a reminder based on the picture taken by the camera. If the learning page to be photographed is not collected, the user is reminded to adjust the position, which can avoid capturing incomplete images and affecting the recognition effect.

[0092] Optionally, if part of the learning page exists in the picture captured by the camera, that is, the learning page is not captured in its entirety by the camera, but only part of it. At this time, the terminal device can determine the direction in which the learning page needs to be moved and the target distance to be moved based on the position of the partial content of the captured learning page compared to the position of the entire picture and the shooting range of the camera, and output accurate prompt information (the prompt information may include the direction in which the learning page needs to be moved and the target distance to be moved) to prompt the user to move the position of the learning page accordingly, so that the user can make the learning page enter the camera's collection range through one accurate movement according to the prompt information.

[0093] For example, if the terminal device only captures the left half of the learning page, it can be determined that the user needs to move the learning page to the left, and based on the position of the left half of the learning page compared to the entire screen and the shooting range of the camera, it is determined that the learning page needs to be moved 10 centimeters. Then a prompt message can be output to the user to prompt the user to move the learning page 10 centimeters to the left, ensuring that the learning page can be captured accurately and clearly.

[0094] This solution determines the precise moving direction and target distance based on the collected learning page, and guides the user to make a single accurate movement so that the camera can collect a clear learning page, thus avoiding the situation where the user still cannot collect a clear learning page after adjusting the position multiple times. This can save the user's operation time and the power consumption of the terminal device.

[0095] 102. Calculate the RGB value of each pixel in the light and shadow.

[0096] In the embodiment of the present invention, there are many pixels in the light and shadow. A pixel is the smallest image unit. Each image is composed of countless pixels. There are different numbers of pixels according to different resolutions. Generally, the higher the resolution, the more pixels. The color of each pixel is represented by an RGB value. The terminal device calculates the RGB value of each pixel in the light and shadow.

[0097] It should be noted that RGB value is a color standard. It is obtained by changing the three primary color channels of red (R), green (G), and blue (B) and superimposing them on each other to obtain various colors. The colors that we can see with our naked eyes are mixed by the three primary colors of red, green, and blue in different proportions. The above RGB value refers to the brightness of the three primary colors. Each light has 256 brightness values, which are expressed as 0, 1, 2...255.

[0098] Alternatively, the RGB color space can be viewed as a unit cube in a three-dimensional rectangular coordinate color system, with the three axes representing the three primary colors of red, green, and blue. Any color that we can see with our naked eyes can be represented by a point in the three-dimensional space in the RGB color space.

[0099] Exemplarily, as shown in Table 1, is the correspondence between different colors and different RGB values.

[0100] Table 1

[0101] Color Name Red value red Green value green blue valueblue black 0 0 0 blue 0 0 255 green 0 255 0 blue 0 255 255 red 255 0 0 Magenta 255 0 255 yellow 255 255 0 White 255 255 255

[0102] It can be seen that in the RGB color space, when the brightness value of the three primary colors is zero, that is, at the origin, it appears black. When the three primary colors reach the highest brightness, it appears white. In light and shadow, due to the divergence of light, the color of the center and edge of the light and shadow will be uneven, that is, there is a difference between the RGB value of the center and the RGB value of the edge of the light and shadow.

[0103] 103. Determine the pixel with the largest RGB value as the center of gravity of the brightness value of light and shadow.

[0104] It should be noted that when comparing the RGB values ​​of each pixel, the three components in the RGB value can be weighted to obtain a processed RGB value. The processed RGB value can also be generally referred to as a grayscale value. In an embodiment of the present invention, the terminal device can perform weighted processing on the three components in the RGB value of each pixel in the light and shadow, obtain the processed RGB value of each pixel and compare them, and determine the pixel with the largest RGB value as the center of gravity of the brightness value of the light and shadow.

[0105] For example, blue, green and cyan are relatively close, and it is difficult to say which color has the largest RGB value. In this case, the three components in the RGB value of each color can be weighted. The weighted formula is RGB value = R*0.3+G*0.59+B*0.11. Substituting the three components of the RGB values ​​of the three colors into the calculation, the RGB value of blue is 28.05, the RGB value of green is 150.45, and the RGB value of cyan is 178.5. At this time, it can be determined that the RGB value of cyan is the largest.

[0106] In an embodiment of the present invention, when taking a photo through a camera, if the camera is facing the learning page to be photographed, that is, the optical axis is perpendicular to the learning page to be photographed, then the pixel with the largest RGB value is the center point of the light and shadow. However, in reality, when the user takes a photo, the camera will have a certain tilt angle, which causes the optical axis and the learning page to be photographed to be not perpendicular. Then the light and shadow presented on the learning page will be deformed compared to the projected light and shadow, and the RGB value of the light and shadow presented near the camera will be larger than the RGB value of the light and shadow presented far from the camera. At this time, the pixel with the largest RGB value is not the center point of the light and shadow.

[0107] 104. Determine the optical axis based on the center of gravity of the light and shadow projector and the brightness value.

[0108] In the embodiment of the present invention, Figure 1b As shown, through the above operation, the pixel point with the largest RGB value in the light and shadow 15 projected on the learning page 14 to be photographed is determined as the brightness value center point 16. The light and shadow projector 13 set in the camera 12 connected to the terminal device 11 and the brightness value center point 16 form an optical axis 17, which is essentially a line in the light projected by the light and shadow projector, and is a virtual non-existent line.

[0109] 105. Project the first image onto a plane perpendicular to the optical axis using perspective transformation technology to obtain a target image.

[0110] Among them, perspective transformation refers to the transformation that uses the condition that the perspective center, image point, and target point are collinear, and according to the perspective rotation law, the perspective plane is rotated around the perspective axis by a certain angle, destroying the original projection light beam, while still keeping the projection geometry on the perspective plane unchanged. The essence of perspective transformation is to project the image onto a new viewing plane, and there is a general transformation formula:

[0111]

[0112] It should be noted that the transformation formula can be applied to a two-dimensional plane or a three-dimensional space. In the embodiment of the present invention, it is a two-dimensional plane transformation in a three-dimensional space. Where (u, v) is the pixel coordinate of the original image before correction, and w can be regarded as a fixed parameter and does not participate in the calculation. is the corrected image pixel coordinate. Transformation matrix It can be divided into four parts, including: represents the linear transformation of the image, Used to generate image perspective transformation, T3 = [a 31 a 32 ] represents image translation. At this time, the expression of perspective transformation can be written as the following formula:

[0113]

[0114]

[0115] At this time, the known pixel coordinates of the four original images before correction can be put into the expression to obtain the transformation formula. Then, each pixel point in the first image is converted into a new pixel point through the transformation formula to obtain the target image.

[0116] The embodiment of the present invention provides an image conversion method. The terminal device can project light and shadow on the learning page to be photographed and collect images including the light and shadow. The pixel point with the largest brightness value in the light and shadow is determined as the center of gravity of the brightness value. At this time, the optical axis can be obtained. Because the camera of the terminal device and the learning page to be photographed are not parallel, there is a certain angle between the optical axis and the collected image, so that the image will be deformed. Finally, the collected image is projected onto the vertical plane of the optical axis through the perspective transformation technology. At this time, the image and the optical axis can be guaranteed to be completely vertical, and the image without deformation after correction can be obtained. In this way, not only can the content required by the user be identified in a targeted manner, but also the accuracy of image recognition and text recognition can be improved.

[0117] Optionally, when there is a visualized straight line in the light and shadow, after collecting the first image including the light and shadow, the method may further include: obtaining the coordinates of the two endpoints of the visualized straight line and the coordinates of the midpoint of the straight line. The first slope of the straight line where the first endpoint and the midpoint are located and the second slope of the straight line where the second endpoint and the midpoint are located are calculated respectively, and when the difference between the first slope and the second slope is greater than a preset difference, a prompt message is output, which is used to prompt the user that the learning page to be photographed is not flat.

[0118] It should be noted that the existence of a visual straight line in the light and shadow means that there is a straight line in the outline of the light and shadow, for example, the light and shadow is a square or rectangle, and its outline is a straight line edge; or the light and shadow is a cross shape, which is composed of two straight lines. When there is a visual straight line in the light and shadow, if the learning page to be photographed is placed flat, then the light and shadow presented on the learning page to be photographed should also have a straight line; but if the learning page to be photographed is not flat, such as an arched book page that is rolled up, then the outline of the light and shadow presented on the learning page to be photographed will change, and the original straight line will become a curve after projection.

[0119] Exemplarily, assuming that the light and shadow are in the shape of a cross, the preset difference is set based on the accuracy of the light and shadow projector and the actual shooting environment. Assume that the preset difference is 0.2. When the light and shadow projector projects the cross-shaped light and shadow on the learning page to be photographed, take three points on one of the edges, the left endpoint A, the intersection O and the right endpoint B. After the coordinate axis is established, the coordinates of the three points are obtained: A(5, 2), O(6, 5), B(7, 6). At this time, the slopes of AO and BO are calculated, k AO =(5-2) / (6-5)=3, k BO =(6-5) / (7-6)=1, the difference between the two slopes is k AO -k BO =2, which is greater than the preset difference, it can be said that the learning page to be photographed is not flat. At this time, the terminal device outputs a prompt message, which is used to suggest the user to place the learning page flat.

[0120] This optional technical solution calculates the shape of the visual straight line in the light and shadow presented on the learning page to be photographed. If the shape presented on the learning page to be photographed is not a straight line, it can be said that the learning page to be photographed is not flat, and then a corresponding reminder is output. This technical solution can prevent the user from placing the learning page unevenly, resulting in the unrecognizable collected image, and saves power consumption of the terminal device.

[0121] Embodiment 2

[0122] like Figure 2 As shown, the image conversion method provided by the embodiment of the present invention may also include the following steps:

[0123] 201. Collect a first image and a second image.

[0124] The first image includes the light and shadow projected by the light and shadow projector, and the second image does not include the light and shadow projected by the light and shadow projector.

[0125] The terminal device can capture an image including light and shadow after the light projector projects the light and shadow, and then turn off the light and shadow projector within a preset time and capture an image that does not include light and shadow, wherein the preset time is set by the terminal device itself, and is generally a relatively small value, such as 0.1s, 0.2s.

[0126] 202. Obtain an RGB value of each pixel in the first image and an RGB value of each pixel in the second image.

[0127] The terminal device analyzes the first image and the second image respectively to obtain the RGB value of each pixel in the two images.

[0128] It can be understood that there are i pixels in the first image, marked as X1, X2, ... i ; There are also i pixels in the second image, marked as Y1, Y2, ...Y i . Each pixel will have an RGB value.

[0129] 203. Perform difference analysis on the RGB value of each corresponding pixel in the first image and the second image.

[0130] In the embodiment of the present invention, the terminal device processes the two images by using image subtraction, and subtracts the RGB values ​​of the corresponding pixels in the two images to obtain the difference of the GRB value of each pixel.

[0131] It can be understood that there are i pixels in the first image, marked as X1, X2, ... i ; There are also i pixels in the second image, marked as Y1, Y2, ...Y i Subtract the pixels in the first image from the second image, and replace X1, X2, ... i and Y1, Y2…Y i In the order of arrangement, subtract the RGB value of X1 from the RGB value of Y1, the RGB value of X2 from the RGB value of Y2, and so on. i Subtract Y from the RGB value i Thus, the difference of i RGB values ​​can be obtained.

[0132] 204. Determine light and shadow according to all pixel points in the first image whose RGB value differences are greater than a preset value.

[0133] The difference of the i RGB values ​​obtained above is compared with the preset value, and all the pixels greater than the preset value are the pixels that constitute the light and shadow.

[0134] Exemplarily, the preset value is set by the terminal device according to the brightness value of the light and shadow transmitted by the light and shadow projector, and it is assumed that the preset value is 50. After performing difference analysis on the RGB value of each corresponding pixel in the first image and the second image, the difference of the RGB values ​​of 20 pixels among the obtained i RGB value differences is greater than 50, then it can be determined that these 20 pixels constitute the light and shadow.

[0135] 205. Calculate the RGB value of each pixel in the light and shadow.

[0136] 206. Determine the pixel point with the largest RGB value as the center of gravity of the brightness value of the light and shadow.

[0137] 207. Determine the optical axis according to the center of gravity of the light and shadow projector and the brightness value.

[0138] 208. Project the first image onto a plane perpendicular to the optical axis using perspective transformation technology to obtain a target image.

[0139] In the embodiment of the present invention, for the description of steps 205 to 208, please refer to the detailed description of steps 102 to 105 in the first embodiment, and the embodiment of the present invention will not be repeated here.

[0140] The embodiment of the present invention provides an image conversion method, which can obtain light and shadow by collecting an image with light and shadow and an image without light and shadow, performing difference analysis on the pixels in each image, and then determining the center of gravity of the brightness value and the optical axis, and projecting the collected image onto a plane perpendicular to the optical axis through perspective transformation technology. This technical solution can obtain accurate light and shadow, and can ensure that the image is completely perpendicular to the optical axis, obtain an image without deformation, and improve the accuracy of image recognition and text recognition.

[0141] As an optional implementation method, before capturing the first image including light and shadow, an instruction from a user can also be accepted. The instruction is used to trigger the terminal device to control the camera to capture the image. The instruction can be implemented in various forms, such as the user clicking on the virtual controls or physical buttons in the terminal device, or it can be a voice instruction input by the user regarding taking pictures.

[0142] Among them, the terminal device controlling the camera to collect images includes the terminal device controlling the light and shadow projector to project light and shadow onto the learning page to be photographed.

[0143] Furthermore, the terminal device responds to the user's first instruction and can also record the activity trajectory of the user's finger through the camera. If the terminal device detects that the user's finger remains in a certain area on the learning page for a preset period of time, the camera is controlled to adjust the shooting angle so that the light and shadow projector can project light and shadow to the area where the finger is located in the learning page to be photographed, and the camera is controlled to capture the image.

[0144] For example, the preset time length can be a time length set by the terminal device or set by the user. Assume that the preset time length is 8 seconds. If the terminal device detects that the user has been pointing at the volume formula of the triangular pyramid “V=1 / 3πr 2 h(the volume of a triangular pyramid is equal to the base area multiplied by the height divided by three, and the base area is equal to π multiplied by the square of the radius)", then the terminal device can control the camera to adjust the shooting angle so that the light and shadow projector can project light and shadow to the area where the triangular pyramid volume formula is located in the learning page to be photographed, and control the camera to collect the image of the area where the triangular pyramid volume formula is located.

[0145] If the user keeps their finger on a certain area for a long time, it is very likely that they have encountered difficulties in the learning process. This implementation method can control the light and shadow projector to project light and shadow on the area where the user's finger is located and collect images when the user's touch operation duration is longer than the preset duration, helping the user solve the problems encountered in the learning process.

[0146] Embodiment 3

[0147] like Figure 3 As shown, the image conversion method provided by the embodiment of the present invention may also include the following steps:

[0148] 301. Collect a first image including light and shadow.

[0149] 302. Calculate the RGB value of each pixel in the light and shadow.

[0150] 303. Determine the pixel point with the largest RGB value as the center of gravity of the brightness value of the light and shadow.

[0151] 304. Determine the optical axis according to the center of gravity of the light and shadow projector and the brightness value.

[0152] In the embodiment of the present invention, for the description of steps 301 to 304, please refer to the detailed description of steps 101 to 104 in the first embodiment, and the embodiment of the present invention will not be repeated here.

[0153] 305. Determine a vertical plane of the optical axis through the target pixel point.

[0154] The target pixel is any pixel in the first image.

[0155] It should be noted that a three-dimensional space coordinate system needs to be established to determine the vertical plane. Take any pixel point as the coordinate origin and establish a three-dimensional rectangular coordinate system, then you can get the coordinate value of the target pixel point and the equation of the line where the optical axis is located. At this time, draw a vertical line from the target pixel point to the optical axis, and you can get the intersection of the vertical line and the line where the optical axis is located. Through the intersection, draw another straight line that is perpendicular to the line where the optical axis is located and is not parallel to the vertical line from the target pixel point to the optical axis. At this time, the plane where these two straight lines that are perpendicular to the line where the optical axis is located and intersect and are not parallel can be determined as the vertical plane of the optical axis.

[0156] 306. Project all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points.

[0157] In the embodiment of the present invention, after the vertical plane of the optical axis is obtained, all the pixel points in the first image can be projected onto the vertical plane to obtain the coordinates of the new pixel points after projection.

[0158] Optionally, all pixel points in the first image are projected onto a plane perpendicular to the optical axis, specifically including: establishing a circumscribed rectangle of light and shadow to obtain N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4; establishing a coordinate axis through the target pixel point to obtain the first coordinate values ​​of the N tangent points; substituting the first coordinate values ​​of the N tangent points into a perspective change expression to calculate a first perspective transformation matrix; multiplying the first coordinate values ​​of all pixel points in the first image by the first perspective transformation matrix to obtain the second coordinate values ​​of all pixel points.

[0159] It should be noted that all the pixels in the first image are projected onto the vertical plane of the optical axis. Specifically, the perspective transformation matrix is ​​determined through at least four tangent points in the first image, and then the first image as a whole is transformed according to the perspective transformation matrix. If the light and shadow presented on the learning page is elliptical, then the elliptical light and shadow will have four tangent points with the circumscribed rectangle. By establishing a coordinate axis through one of the tangent points, the coordinates of the four tangent points can be obtained: A1 (u0, v0), A2 (u1, v1), A3 (u2, v2), A4 (u3, v3). Substitute the coordinates of these four points into the perspective change expression

[0160]

[0161] It should be noted that the transformation formula can be applied to a two-dimensional plane or a three-dimensional space. In the embodiment of the present invention, it is a two-dimensional plane transformation in a three-dimensional space. Where (u, v) is the pixel coordinate of the original image before correction, and w can be regarded as a fixed parameter and does not participate in the calculation. is the corrected image pixel coordinate. 33 After determining it as 1 and substituting the coordinates of the four tangent points, we can get the following equation:11 x+a 12 y+a 13 -a 31 xX-a 32 yX=X,a 21 x+a 22 y+a 23 -a 31 xY 32 yY=Y, where By bringing in the coordinates of the four tangent points, we can calculate the perspective transformation matrix At this time, the coordinates of each pixel point in the first image are multiplied by the perspective transformation matrix A to obtain the transformed second coordinate value.

[0162] This optional technical solution establishes a circumscribed rectangle of light and shadow to obtain at least four tangent points, and then calculates the perspective transformation matrix based on the four tangent points to perform image transformation. At this time, the image and the optical axis can be guaranteed to be completely perpendicular, and a corrected image without deformation can be obtained. This can improve the accuracy of image recognition and text recognition.

[0163] 307. Determine a third image according to the projected pixel points and the target pixel points.

[0164] According to the obtained converted second coordinate value of each pixel point, combined with the target pixel point, a third image composed of all the pixels can be obtained.

[0165] 308. Perform smoothing processing on all pixels in the third image to obtain a target image.

[0166] In the embodiment of the present invention, the terminal device usually uses a low-pass filtering method to smooth the third image so as to eliminate noise and obtain a target image.

[0167] Image signals are interfered by various noises during generation and transmission, so the images obtained are all noisy images. Generally, smoothing is required before image segmentation and extraction. Generally speaking, image smoothing is mainly to eliminate noise. Noise is not limited to distortion and deformation visible to the human eye. Some noise can only be found during image processing. Common image noises are mainly additive noise, multiplicative noise, and quantization noise. Since the energy of the image is mainly concentrated in the low-frequency part, and the frequency band where the noise is located is mainly in the high-frequency band, low-pass filtering is usually used to eliminate noise. Common filters include mean filtering, median filtering, Gaussian filtering, etc.

[0168] The embodiment of the present invention provides an image conversion method, where a terminal device can project light and shadow on a learning page to be photographed and collect an image including the light and shadow, determine the pixel point with the largest brightness value in the light and shadow as the center of gravity of the brightness value, and then obtain the optical axis, project the first image onto a plane perpendicular to the optical axis, and smooth the projected pixel points to obtain the final target image. This technical solution can ensure that the image is completely perpendicular to the optical axis, obtain an image that is not deformed, and improve the accuracy of image recognition and text recognition.

[0169] As an optional implementation method, after obtaining the target image, the terminal device displays the target image on the display screen of the terminal device. The terminal device can also identify the content in the target image, search the identification result in a pre-stored database, and display the search result on the display screen of the terminal device other than the target image.

[0170] Furthermore, a first control may be displayed on the display screen of the terminal device, and the terminal device switches from displaying the target image and the search results to displaying the target image in response to a user's touch input to the first control. The terminal device may also switch from displaying the target image to displaying the target image and the search results in response to a second touch input to the first control by the user.

[0171] In this optional implementation, the terminal device can set a control in the display page, and after searching for the target image, the search results are displayed on the display screen. If the user clicks the control, the target image without the search results will be displayed, that is, the search results will be hidden. If the user clicks the control again, the search results will be displayed again. This technical solution can search for the target image and hide the search results when they are not needed, which can meet the different needs of users.

[0172] Embodiment 4

[0173] like Figure 4 As shown, an embodiment of the present invention provides a terminal device, the terminal device comprising:

[0174] The acquisition module 401 is used to acquire a first image including light and shadow.

[0175] The processing module 402 is used to calculate the RGB value of each pixel in the light and shadow; and to project the first image onto a plane vertical to the optical axis through a perspective transformation technique to obtain a target image.

[0176] The determination module 403 is used to determine the pixel point with the largest RGB value as the center of gravity of the brightness value of the light and shadow; and to determine the optical axis according to the light and shadow projector and the center of gravity of the brightness value.

[0177] Optionally, the acquisition module 401 is further used to capture the first image and the second image, the second image does not include light and shadow; and to acquire the RGB value of each pixel in the first image and the RGB value of each pixel in the second image;

[0178] The processing module 402 is further used to perform difference analysis on the RGB value of each corresponding pixel point in the first image and the second image;

[0179] The determination module 403 is further configured to determine light and shadow according to all pixel points in the first image whose RGB value differences are greater than a preset value.

[0180] Optionally, the determination module 403 is further used to determine a vertical plane of the optical axis through a target pixel point, wherein the target pixel point is any pixel point in the first image; and to determine a third image based on the projected pixel point and the target pixel point.

[0181] The processing module 402 is further used to project all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points; and to perform smoothing processing on all pixel points in the third image to obtain a target image.

[0182] Optionally, the processing module 402 is also used to establish a circumscribed rectangle of the light and shadow, and obtain N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4; and to establish a first coordinate axis through a target pixel point, and obtain first coordinate values ​​of the N tangent points, where the target pixel point is specifically any one of the N tangent points; and to substitute the first coordinate values ​​of the N tangent points into a perspective change expression to calculate a first perspective transformation matrix; and to multiply the first coordinate values ​​of all pixels in the first image by the first perspective transformation matrix to obtain second coordinate values ​​of all pixels.

[0183] Optionally, when there is a visualized straight line in the light and shadow, the acquisition module 401 is further used to acquire the coordinate value of the first endpoint of the visualized straight line, the coordinate value of the second endpoint of the visualized straight line, and the coordinate value of the midpoint between the first endpoint and the second endpoint;

[0184] The processing module 402 is further used to calculate a first slope of a straight line where the first endpoint and the midpoint are located and a second slope of a straight line where the second endpoint and the midpoint are located; and to calculate a difference between the first slope and the second slope;

[0185] The terminal device also includes:

[0186] The output module 404 is used to output a prompt message if the difference is greater than a preset difference, and the prompt message is used to prompt the user that the learning page to be photographed is not flat.

[0187] In the embodiment of the present invention, each module can implement the image conversion method provided by the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0188] like Figure 5 As shown, an embodiment of the present invention further provides a terminal device, which may include:

[0189] A memory 501 storing executable program codes;

[0190] a processor 502 coupled to the memory 501;

[0191] The processor 502 calls the executable program code stored in the memory 501 to execute the image conversion method executed by the terminal device in the above-mentioned method embodiments.

[0192] The terminal device involved in the embodiment of the present invention may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. Among them, the wearable device may be a smart watch, a smart bracelet, a phone watch, a smart anklet, a smart earring, a smart necklace, a smart headset, etc., which is not limited in the embodiment of the present invention.

[0193] like Figure 6 As shown, the embodiment of the present invention further provides a terminal device, which includes but is not limited to: a radio frequency (RF) circuit 601, a memory 602, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a WiFi (wireless fidelity) module 607, a processor 608, a power supply 609, and a camera 610. Among them, the RF circuit 601 includes a receiver 6011 and a transmitter 6012. Those skilled in the art can understand that Figure 6 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0194] The RF circuit 601 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 608 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 601 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 601 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0195] The memory 602 can be used to store software programs and modules. The processor 608 executes various functional applications and data processing of the terminal device by running the software programs and modules stored in the memory 602. The memory 602 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the terminal device (such as audio data, a phone book, etc.), etc. In addition, the memory 602 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0196] The input unit 603 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the terminal device. Specifically, the input unit 603 may include a touch panel 6031 and other input devices 6032. The touch panel 6031, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 6031 or near the touch panel 6031 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 6031 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 608, and can receive and execute the command sent by the processor 608. In addition, the touch panel 6031 can be implemented in a variety of ways such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 6031, the input unit 603 may also include other input devices 6032. Specifically, the other input devices 6032 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0197] The display unit 604 can be used to display information input by the user or information provided to the user and various menus of the terminal device. The display unit 604 may include a display panel 6041. Optionally, the display panel 6041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 6031 may cover the display panel 6041. When the touch panel 6031 detects a touch operation on or near it, it is transmitted to the processor 608 to determine the touch event. Subsequently, the processor 608 provides a corresponding visual output on the display panel 6041 according to the touch event. Although in Figure 6 In the embodiment, the touch panel 6031 and the display panel 6041 are used as two independent components to realize the input and output functions of the terminal device, but in some embodiments, the touch panel 6031 and the display panel 6041 can be integrated to realize the input and output functions of the terminal device.

[0198] The terminal device may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 6041 according to the brightness of the ambient light, and the proximity sensor may exit the display panel 6041 and / or the backlight when the terminal device is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the terminal device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can be configured in the terminal device, they will not be repeated here. In an embodiment of the present invention, the terminal device may include an accelerometer, a depth sensor, or a distance sensor, etc.

[0199] The audio circuit 606, the speaker 6061, and the microphone 6062 can provide an audio interface between the user and the terminal device. The audio circuit 606 can transmit the received audio data to the speaker 6061 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 6062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 606 and converted into audio data, and then the audio data is processed by the output processor 608, and then sent to another terminal device through the RF circuit 601, or the audio data is output to the memory 602 for further processing.

[0200] WiFi is a short-range wireless transmission technology. The terminal device can help users send and receive emails, browse web pages and access streaming media through the WiFi module 607. It provides users with wireless broadband Internet access. Figure 6 A WiFi module 607 is shown, but it is understandable that it is not a necessary component of the terminal device and can be omitted as needed without changing the essence of the invention.

[0201] The processor 608 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal device. By running or executing software programs and / or modules stored in the memory 602 and calling data stored in the memory 602, it executes various functions of the terminal device and processes data, thereby monitoring the terminal device as a whole. Optionally, the processor 608 may include one or more processing units; preferably, the processor 608 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 608.

[0202] The terminal device also includes a power supply 609 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 608 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. Although not shown, the terminal device can also include a Bluetooth module, etc., which will not be repeated here.

[0203] In the embodiment of the present invention, the processor 608 is used to collect a first image including light and shadow;

[0204] Calculate the RGB value of each pixel in the light and shadow, and determine the pixel with the largest RGB value as the center of gravity of the brightness value of the light and shadow;

[0205] Determine the optical axis based on the center of gravity of the light and shadow projector and the brightness value;

[0206] The first image is projected onto a plane perpendicular to the optical axis using perspective transformation technology to obtain a target image.

[0207] Optionally, the processor 608 may also be used to implement other processes implemented by the terminal device in the method embodiment.

[0208] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps of the method in the above method embodiments.

[0209] An embodiment of the present invention further provides a computer program product, wherein when the computer program product runs on a computer, the computer is enabled to execute part or all of the steps of the methods in the above method embodiments.

[0210] An embodiment of the present invention further provides an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of the method in the above method embodiments.

[0211] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0212] The terminal device provided in the embodiment of the present invention can implement each process shown in the above method embodiment, and will not be described again here to avoid repetition.

[0213] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0214] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0216] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0217] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

Claims

1. An image conversion method, characterized in that: Applied to a terminal device, a camera of the terminal device is provided with a light and shadow projector, and the light and shadow projected by the light and shadow projector is within the acquisition range of the camera, including: The movement track of the user's finger is recorded by the camera. If it is detected that the user's finger has been staying in a certain area of ​​the learning page to be photographed within a preset time, the camera is controlled to adjust the shooting angle so that the light and shadow projector projects light and shadow to the area where the finger is located in the learning page to be photographed, so as to collect a first image including the light and shadow and a second image not including the light and shadow, and the shape of the light and shadow can be adjusted according to the content of the learning page to be photographed; Obtaining the RGB value of each pixel in the first image and the RGB value of each pixel in the second image; Performing difference analysis on the RGB values ​​of each corresponding pixel in the first image and the second image, and determining the light and shadow according to the pixels in the first image whose differences in RGB values ​​are greater than a preset value; Calculate the RGB value of each pixel in the light and shadow, and determine the pixel with the largest RGB value as the center of gravity of the brightness value of the light and shadow; Determine the optical axis according to the light and shadow projector and the center of gravity of the brightness value; The first image is projected onto a plane vertical to the optical axis by using a perspective transformation technique to obtain a target image.

2. The method according to claim 1, characterized in that The projecting the first image onto a plane vertical to the optical axis by using a perspective transformation technique to obtain a target image comprises: Determining a vertical plane of the optical axis through a target pixel point, wherein the target pixel point is any pixel point in the first image; Projecting all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points; Determine a third image according to the projected pixel points and the target pixel points; All pixels in the third image are smoothed to obtain the target image.

3. The method according to claim 2, characterized in that The step of projecting all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points includes: Establishing a circumscribed rectangle of the light and shadow, and obtaining N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4; Establishing a first coordinate axis through the target pixel point, obtaining first coordinate values ​​of the N tangent points, wherein the target pixel point is specifically any one of the N tangent points; Substituting the first coordinate values ​​of the N tangent points into the perspective change expression to calculate a first perspective transformation matrix; The first coordinate values ​​of all pixels in the first image are multiplied by the first perspective transformation matrix to obtain second coordinate values ​​of all pixels.

4. The method according to any one of claims 1 to 3, characterized in that: In the case where there is a visualized straight line in the light and shadow, after acquiring the first image including the light and shadow and the second image not including the light and shadow, the method further includes: Acquire the coordinate value of the first endpoint of the visualized straight line, the coordinate value of the second endpoint of the visualized straight line, and the coordinate value of the midpoint between the first endpoint and the second endpoint; Calculate a first slope of a straight line between the first endpoint and the midpoint and a second slope of a straight line between the second endpoint and the midpoint; calculating a difference between the first slope and the second slope; If the difference is greater than a preset difference, a prompt message is output, where the prompt message is used to prompt the user that the learning page to be photographed is not flat.

5. A terminal device, characterized in that: include: An acquisition module is used to record the activity trajectory of the user's finger through a camera. If it is detected that the user's finger has been staying in a certain area of ​​the learning page to be photographed within a preset time, the camera is controlled to adjust the shooting angle so that the light and shadow projector projects light and shadow to the area where the finger is located in the learning page to be photographed, so as to collect a first image including the light and shadow and a second image not including the light and shadow, and the shape of the light and shadow can be adjusted according to the content of the learning page to be photographed; the RGB value of each pixel in the first image and the RGB value of each pixel in the second image are obtained; A processing module, used for performing difference analysis on the RGB value of each corresponding pixel in the first image and the second image; A determination module, configured to determine the light and shadow according to all pixel points in the first image whose RGB value differences are greater than a preset value; The processing module is also used to calculate the RGB value of each pixel in the light and shadow; The determination module is further used to determine the pixel point with the largest RGB value as the center of gravity of the brightness value of the light and shadow; The determination module is further used to determine the optical axis according to the light and shadow projector and the center of gravity of the brightness value; The processing module is further used to project the first image onto a plane vertical to the optical axis through a perspective transformation technique to obtain a target image.

6. The terminal device according to claim 5, characterized in that: The determination module is further used to determine a vertical plane of the optical axis through a target pixel point, wherein the target pixel point is any pixel point in the first image; The processing module is further used to project all pixel points in the first image onto a plane perpendicular to the optical axis to obtain projected pixel points; The determination module is further used to determine a third image according to the projected pixel points and the target pixel points; The processing module is further used to perform smoothing processing on all pixels in the third image to obtain the target image.

7. The terminal device according to claim 6, characterized in that: The processing module is further used to establish a circumscribed rectangle of the light and shadow, and obtain N tangent points between the light and shadow and the circumscribed rectangle, where N is an integer greater than or equal to 4; The processing module is further used to establish a first coordinate axis through the target pixel point to obtain first coordinate values ​​of the N tangent points, wherein the target pixel point is specifically any one of the N tangent points; The processing module is further used to substitute the first coordinate values ​​of the N tangent points into the perspective change expression to calculate a first perspective transformation matrix; The processing module is further used to multiply the first coordinate values ​​of all pixels in the first image by the first perspective transformation matrix to obtain second coordinate values ​​of all pixels.

8. The terminal device according to any one of claims 5 to 7, characterized in that: In the case where there is a visual straight line in the light and shadow, The acquisition module is further used to acquire the coordinate value of the first endpoint of the visualized straight line, the coordinate value of the second endpoint of the visualized straight line, and the coordinate value of the midpoint between the first endpoint and the second endpoint; The processing module is further used to calculate a first slope of a straight line where the first endpoint and the midpoint are located and a second slope of a straight line where the second endpoint and the midpoint are located; The processing module is further used to calculate the difference between the first slope and the second slope; The terminal device further includes: The output module is used to output a prompt message if the difference is greater than a preset difference, wherein the prompt message is used to prompt the user that the learning page to be photographed is not flat.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the image conversion method according to any one of claims 1 to 4 is implemented.

10. A computer program product, characterized in that include: The computer program product comprises computer instructions. When the computer program product runs on a processor, the processor executes the computer instructions to implement the image conversion method according to any one of claims 1 to 4.

Images