Image processing method, device, electronic device and storage medium

By filtering and using the key point feature information in the image processing method to splice the multi-frame images scanned by the scanning pen, the problem of splicing effect under the influence of scanning angle and movement speed is solved, and the accuracy of text recognition is improved.

CN114926345BActive Publication Date: 2024-12-06SHANGHAI SENSETIME INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210713849.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-22
Publication Date
2024-12-06
Estimated Expiration
2042-06-22

AI Technical Summary

Technical Problem

During use, existing scanning pens are affected by scanning angle and movement speed, resulting in poor image stitching and inaccurate text recognition results.

Method used

By obtaining multiple frames of images to be processed, the key point feature information is filtered out, the images are spliced ​​based on these feature information, and the stitching image is generated, which reduces interference in the blank area and improves the stitching accuracy.

Benefits of technology

It improves the accuracy of image stitching and the accuracy of recognition results, alleviates the problem of poor stitching effect caused by sparse optical flow or image block matching methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114926345B_ABST
    Figure CN114926345B_ABST
Patent Text Reader

Abstract

The present disclosure provides an image processing method, device, electronic device and storage medium, the method comprising: obtaining multiple frames of images to be processed obtained by continuously scanning text content; for each frame of the images to be processed in the multiple frames, based on the pixel point information of each pixel point in the images to be processed, screening out key points corresponding to objects to be identified in the images to be processed from the pixels; obtaining multiple groups of pixel point pairs within the target area corresponding to the key points, and obtaining key point feature information corresponding to the key points based on the multiple groups of pixel point pairs; based on the key point feature information corresponding to the key points in the multiple frames of images to be processed, splicing the multiple frames of images to be processed to generate a spliced ​​image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the development of information technology and the need for multilingual communication, scanning pens are used in many fields, such as education and social networking. The scanning pen can complete the recognition of subsequent text content by continuously collecting and splicing images during movement. However, the scanning pen is affected by factors such as scanning angle and movement speed during use, which affects the image splicing effect and the final text content recognition result. Summary of the invention

[0003] In view of this, the present disclosure at least provides an image processing method, device, electronic device and storage medium.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] Acquire multiple frames of images to be processed after continuously scanning the text content;

[0006] For each frame of the image to be processed in the plurality of frames of the image to be processed, based on pixel point information of each pixel point in the image to be processed, a key point corresponding to the object to be identified in the image to be processed is screened out from each pixel point;

[0007] Acquire multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs;

[0008] Based on the key point feature information corresponding to each key point in the multiple frames of images to be processed, the multiple frames of images to be processed are spliced ​​to generate a spliced ​​image.

[0009] In the above method, for each frame of the image to be processed in the multiple frames of the image to be processed, based on the pixel information of each pixel in the image to be processed, the key point corresponding to the object to be identified in the image to be processed is screened out from each pixel; for example, when the object to be identified includes Chinese characters, the key point can be a pixel corresponding to the boundary of the Chinese character, or can be a pixel corresponding to the blank area around the boundary of the Chinese character. Then, multiple groups of pixel pairs corresponding to the key point can be obtained, and key point feature information corresponding to the key point can be obtained based on the multiple groups of pixel pairs; because the key point can be a pixel corresponding to the boundary of the object to be identified, or a pixel corresponding to the blank area around the boundary of the object to be identified, the feature information of the key point is relatively rich, and the key point feature information corresponding to each key point in each frame of the image to be processed can be used to accurately splice the multiple frames of the image to be processed, reduce the interference of other blank areas in the image to be processed except the blank area around the boundary of the object to be identified, and alleviate the problem of poor splicing effect caused by splicing using sparse optical flow or image block matching in the related art, so that the splicing is more accurate and the effect of the generated spliced ​​image is better.

[0010] In a second aspect, the present disclosure provides an image processing device, comprising:

[0011] An acquisition module, used to acquire multiple frames of images to be processed obtained by continuously scanning any text content;

[0012] A screening module, for screening out key points corresponding to the objects to be identified in each frame of the plurality of frames of the images to be processed, from each pixel based on the pixel information of each pixel in the image to be processed;

[0013] A first determination module is used to obtain multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs;

[0014] The first generating module is used to splice the multiple frames of images to be processed based on the key point feature information corresponding to each key point in the multiple frames of images to be processed to generate a spliced ​​image.

[0015] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method described in the first aspect or any one of the embodiments are performed.

[0016] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the image processing method as described in the first aspect or any one of the embodiments above are executed.

[0017] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a flow chart of an image processing method provided by an embodiment of the present disclosure is shown;

[0020] Figure 2 A schematic diagram showing multiple frames of images to be processed obtained in an image processing method provided by an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram showing key points corresponding to an object to be identified in an image processing method provided by an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram showing position information of multiple groups of pixel pairs corresponding to each key point in an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 5 A schematic diagram of stitching images in an image processing method provided by an embodiment of the present disclosure is shown;

[0024] Figure 6 A schematic diagram showing target information in an image processing method provided by an embodiment of the present disclosure;

[0025] Figure 7 A schematic diagram of a target image in an image processing method provided by an embodiment of the present disclosure is shown;

[0026] Figure 8 A schematic diagram of the architecture of an image processing device provided by an embodiment of the present disclosure is shown;

[0027] Fig. 9 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0029] The scanning pen mainly recognizes and plays the content in the multiple frames of images obtained by scanning. Among them, after the scanning pen obtains multiple frames of images, the multiple frames of images scanned by the scanning pen need to be spliced ​​to obtain a spliced ​​image, so as to perform image content recognition based on the spliced ​​image. Image splicing specifically refers to deduplication, alignment and splicing of multiple frames of images generated during the scanning process to construct a complete image. However, due to the scanning angle, the moving speed of the scanning pen and other reasons, it is difficult to align and splice multiple frames of images from the hardware calibration angle to obtain a spliced ​​image.

[0030] In the related art, image stitching can be achieved by registering visual images. In one method, the optical flow method can be used to match and track feature points in multiple frames of images, determine the motion vector, and register the images based on the motion vector to complete the stitching of multiple frames of images. In another method, the motion vector between image pairs can be calculated based on image block matching, and then the image registration can be achieved based on the motion vector to complete the stitching of multiple frames of images.

[0031] However, the above matching method based on sparse optical flow and image blocks performs image matching by dividing the image into multiple blocks. Considering the different sizes of the scanned content, when scanning a small single-line text, such as a single-line size 6 text, except for a small area in the center of the image as the text area, most of the other areas are blank areas, and the matching of blank image blocks is prone to errors, resulting in reduced accuracy of image registration, and thus poor effect of stitching the image. Moreover, even if the image registration is correct, if there is hand shaking or shaking during the scanning process, the text in the stitched image may be curved.

[0032] At the same time, because the camera of the scanning pen is relatively small and flexible and the scanning angles are diverse, the text captured by the camera is easily tilted, that is, the text is not in the horizontal position of the image. When the scanning speed is fast, the motion vector of the scanned images of the previous and next frames is large, and even if the images are aligned, they cannot be spliced. If an additional text angle judgment algorithm is added, the accuracy is poor and the extra time and power consumption are increased, making it difficult to achieve a good splicing effect.

[0033] In order to alleviate the above problems, the embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] To facilitate understanding of the embodiments of the present disclosure, an image processing method disclosed in the embodiments of the present disclosure is first introduced in detail. The execution subject of the image processing method provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be, for example, a mobile phone, a scanning pen, a reading pen, and other devices. In some possible implementations, the image processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0036] See also Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure, and the method includes S101 to S104, specifically:

[0037] S101, obtaining multiple frames of images to be processed after continuously scanning text content.

[0038] S102 , for each frame of the image to be processed in the plurality of frames of the image to be processed, based on pixel information of each pixel in the image to be processed, filtering out key points corresponding to the object to be identified in the image to be processed from each pixel.

[0039] S103, obtaining multiple groups of pixel pairs within the target area corresponding to the key points, and obtaining key point feature information corresponding to the key points based on the multiple groups of pixel pairs.

[0040] S104, based on the key point feature information corresponding to the key points in the multiple frames of images to be processed, stitching the multiple frames of images to be processed to generate a stitched image.

[0041] In the above method, for each frame of the image to be processed in the multiple frames of the image to be processed, based on the pixel information of each pixel in the image to be processed, the key point corresponding to the object to be identified in the image to be processed is screened out from each pixel; for example, when the object to be identified includes Chinese characters, the key point can be a pixel corresponding to the boundary of the Chinese character, or can be a pixel corresponding to the blank area around the boundary of the Chinese character. Then, multiple groups of pixel pairs corresponding to the key point can be obtained, and key point feature information corresponding to the key point can be obtained based on the multiple groups of pixel pairs; because the key point can be a pixel corresponding to the boundary of the object to be identified, or a pixel corresponding to the blank area around the boundary of the object to be identified, the feature information of the key point is relatively rich, and the key point feature information corresponding to each key point in each frame of the image to be processed can be used to accurately splice the multiple frames of the image to be processed, reduce the interference of other blank areas in the image to be processed except the blank area around the boundary of the object to be identified, and alleviate the problem of poor splicing effect caused by splicing using sparse optical flow or image block matching in the related art, so that the splicing is more accurate and the effect of the generated spliced ​​image is better.

[0042] S101 to S104 are described in detail below.

[0043] With respect to S101, a plurality of frames of images to be processed are obtained after continuously scanning the text content:

[0044] The text content may include but is not limited to: text in different languages, numbers, mathematical symbols, etc. Text in different languages ​​may include Chinese characters, English letters, etc.; multiple frames of images to be processed can be obtained by scanning the text content continuously through scanning devices such as scanning pens and point reading pens, and the size of the captured image can be pre-set according to the size of the camera on the scanning device; at the same time, the order of the multiple frames of images to be processed is matched according to the time sequence of the scanning device. Generally, the frame rate of the camera on the scanning device is high, for example, 120FPS, that is, 120 frames of images can be transmitted per second. Therefore, there is an overlapping area between two adjacent frames of images to be processed, see Figure 2 In the five frames of images shown, there is partial overlap between two adjacent frames.

[0045] For S102, for each frame of the image to be processed in the multiple frames of the image to be processed, based on the pixel information of each pixel in the image to be processed, a key point corresponding to the object to be identified in the image to be processed is screened out from each pixel:

[0046] After obtaining multiple frames of images to be processed, for each frame of the images to be processed, a feature extraction algorithm, such as a surf algorithm, a fast algorithm, etc., can be used to filter out key points corresponding to the object to be identified in the image to be processed from each pixel point included in the image to be processed. The object to be identified can be any object included in the image to be processed, such as text, characters, pictures, etc.; see Figure 3 As shown, taking the object to be identified including Chinese characters as an example, the key point can be the corresponding pixel point on the boundary of the Chinese character, that is, the pixel point with darker color, such as key point 2 and key point 4, or it can be the pixel point corresponding to the blank area around the boundary of the Chinese character, that is, the pixel point with lighter color, which is the pixel point corresponding to the blank area, such as key point 1 and key point 3.

[0047] Specifically, based on the pixel information of each pixel in the image to be processed, the key point corresponding to the object to be identified in the image to be processed can be screened out from each pixel; the key point can be the pixel point corresponding to the boundary of the object to be identified, or can be the pixel point corresponding to the blank area around the boundary of the object to be identified.

[0048] In a possible implementation manner, the step of S102, based on the pixel information of each pixel in the image to be processed, filtering out key points corresponding to the object to be identified in the image to be processed from each pixel, includes:

[0049] Step A1, taking each pixel of at least part of the pixels of the image to be processed as a candidate key pixel, and determining the pixel difference between the pixel information of each boundary pixel on the neighborhood boundary corresponding to the candidate key pixel and the pixel information of the candidate key pixel.

[0050] Step A2: determine the boundary pixel points whose pixel difference is greater than the difference threshold as candidate boundary pixel points.

[0051] Step A3: when the number of candidate boundary pixel points that appear continuously on the neighborhood boundary is greater than a preset number, the candidate key pixel points are determined as key points corresponding to the object to be identified.

[0052] During implementation, each pixel point in at least part of the pixels of the image to be processed is taken as a candidate key pixel point, and the neighborhood corresponding to the candidate key pixel point can be determined first, wherein the neighborhood radius corresponding to the candidate key pixel point can be determined based on the acquired image to be processed, and the size of the image to be processed is small, and the neighborhood radius can be small; for example, if the image resolution of the image to be processed is 80×80, the neighborhood radius can be 15 pixels; if the image resolution of the image to be processed is 128×128, the neighborhood radius can be 30 pixels; and so on; then the pixel difference between the pixel information of each boundary pixel point on the neighborhood boundary corresponding to the candidate key pixel point and the pixel information of the candidate key pixel point can be determined. Furthermore, the boundary pixel point whose pixel difference is greater than the difference threshold can be determined as a candidate boundary pixel point; wherein the difference threshold can be determined according to the actual situation.

[0053] When the number of candidate boundary pixel points that appear continuously on the neighborhood boundary is greater than a preset number, the candidate key point is determined to be the key point corresponding to the object to be identified, and the pixel position of the candidate key point on the image to be processed is determined as the key point position information corresponding to the key point; when the number of candidate boundary pixel points that appear continuously on the neighborhood boundary is less than or equal to the preset number, the candidate key point is determined not to be a key point; wherein the preset number can be determined based on the number of boundary pixel points, and the number of boundary pixel points is positively correlated with the preset number, that is, the more the number of boundary pixel points, the greater the preset number; for example, if the number of boundary pixel points is 30, the preset number can be 18; if the number of boundary pixel points is 20, the preset number can be 10, and so on; the key point position information corresponding to the key point includes information used to reflect the position of the key point in the image to be processed, such as the horizontal and vertical coordinates of the key point on the image to be processed.

[0054] Exemplarily, the number of boundary pixel points corresponding to the candidate key point is 20, and the preset number is 12; the number of candidate boundary pixel points whose pixel difference is greater than the difference threshold is 16, among which there are 12 consecutive candidate boundary pixel points, then it can be determined that the candidate key point is the key point corresponding to the object to be identified.

[0055] Here, each pixel in at least part of the pixel points of the image to be processed can be first taken as a candidate key point, and then the key point and key point position information can be more accurately determined based on the pixel information of each boundary pixel point on the neighborhood boundary corresponding to the candidate key point and the pixel information of the candidate key point, so that the key feature information of the key point can be used for image stitching later.

[0056] For S103, multiple groups of pixel pairs within the target area corresponding to the key point are obtained, and key point feature information corresponding to the key point is obtained based on the multiple groups of pixel pairs:

[0057] After determining the key points corresponding to the objects to be identified in each frame of the image to be processed, a feature extraction sub-description algorithm, such as the brief algorithm and the star algorithm, can be used to generate a binary description corresponding to the key points, that is, key feature information.

[0058] Specifically, multiple groups of pixel pairs within the target area corresponding to the key points can be obtained first, and key point feature information corresponding to the key points can be obtained based on the multiple groups of pixel pairs; the key point feature information includes pixel deviation information of multiple groups of pixel pairs within the target area corresponding to the key points; wherein the number of pixel pairs can be set in advance, and the accuracy of the key point feature information is proportional to the number of pixel pairs, for example, the more pixel pairs are set, the higher the accuracy of the key point feature information.

[0059] During implementation, the shape of the target area corresponding to the key point can be a regular shape or an irregular shape; if the shape of the target area is a regular shape, the target area can be a circular area, a square area, etc.; at the same time, the position of the key point within the corresponding target area can be the center position or other positions. For example, if the target area is a square area, the key point can be located at any vertex position of the square area; the specific position can be determined according to actual needs; but it is necessary to ensure that the target area range corresponding to each key point is consistent, and the position information of each key point within the corresponding target area is consistent.

[0060] Taking the target area including a circular area as an example, multiple sets of pixel pairs within the target area corresponding to the key points are obtained, and key point feature information corresponding to the key points is obtained based on the multiple sets of pixel pairs, including:

[0061] Step B1, determine the target area range with the key point as the center and a preset length as the radius.

[0062] Step B2: determining multiple groups of pixel pairs corresponding to key points within the target area.

[0063] Step B3: for each pixel point pair in the plurality of pixel point pairs, determine pixel deviation information between pixel information of a first pixel point and pixel information of a second pixel point in the pixel point pair.

[0064] Step B4: generating key point feature information corresponding to the key points based on the obtained pixel deviation information corresponding to the plurality of groups of pixel point pairs.

[0065] During implementation, when the target area includes a circular area, the target area with the key point as the center and the preset length as the radius is first determined; wherein the preset length can be determined according to the actual situation, for example, it can be 12 pixels, 20 pixels, etc. Then, multiple groups of pixel pairs corresponding to the key points can be determined within the target area; the number of pixel pairs can be pre-set according to the actual situation, for example, the number of preset pixel pairs is 10 pairs, and 20 pixels can be randomly selected within the target area, pixel 1 to pixel 20, and then 10 groups of pixel pairs can be formed according to the selection order, that is, pixel 1 and pixel 2, pixel 3 and pixel 4, pixel 5 and pixel 6, ..., pixel 19 and pixel 20.

[0066] Furthermore, for each group of pixel pairs in the multiple groups of pixel pairs, pixel deviation information between pixel information of the first pixel point in the pixel pair and pixel information of the second pixel point in the pixel pair can be determined; wherein the first pixel point and the second pixel point in the pixel pair can be randomly determined.

[0067] For example, if the pixel information of the first pixel is greater than the pixel information of the second pixel, the pixel deviation information between the first pixel and the second pixel is 1; if the pixel information of the first pixel is less than the pixel information of the second pixel, the pixel deviation information between the first pixel and the second pixel is 0. Of course, the opposite mechanism can also be used when assigning values, that is, if the pixel information of the first pixel is greater than the pixel information of the second pixel, the pixel deviation information between the first pixel and the second pixel is 0; if the pixel information of the first pixel is less than the pixel information of the second pixel, the pixel deviation information between the first pixel and the second pixel is 1. For the above two implementation methods, if the pixel information of the first pixel is equal to the pixel information of the second pixel, the pixel deviation information between the first pixel and the second pixel can be 1 or 0.

[0068] Taking the first assignment mechanism mentioned above, there are 10 groups of pixel pairs as an example (i.e., including pixel 1 to pixel 20), the pixel information of pixel 1 is 198, and the pixel information of pixel 2 is 150, then the pixel deviation information between pixel 1 and pixel 2 can be 1 (198>150); the pixel information of pixel 3 is 28, and the pixel information of pixel 4 is 88, and the pixel deviation information between pixel 3 and pixel 4 can be 0 (28<88); the pixel information of pixel 5 is 200, and the pixel information of pixel 6 is 200. The pixel information of pixel 6 is 200, and the pixel deviation information between pixel 5 and pixel 6 can be 1; ...; the pixel information of pixel 19 is 100, the pixel information of pixel 20 is 120, and the pixel deviation information between pixel 19 and pixel 20 can be 0 (100<120); based on the obtained pixel deviation information corresponding to multiple groups of pixel pairs, key point feature information corresponding to the key points can be generated, for example, the key point feature information can be expressed as 1011101000.

[0069] Considering that the key point matching pairs will be determined based on the key point feature information corresponding to the key points, in order to ensure the accuracy of the key point matching pairs, combined with Figure 4 As shown, within the target area corresponding to each key point, when determining multiple groups of pixel pairs corresponding to the key point, a coordinate system can be established with the center point of the target area corresponding to each key point as the origin. For each target area, the pixel located at position information (-5, 6) is determined to be pixel a, the pixel located at position information (5, 2) is pixel b, the pixel located at position information (4, 7) is pixel c, the pixel located at position information (-2, -5) is pixel d, the pixel located at position information (-6, -1) is pixel e, and the pixel located at position information (5, 3) is pixel f; then pixel a and pixel b constitute pixel pair 1, pixel c and pixel d constitute pixel pair 2, and pixel e and pixel f constitute pixel pair 3. It can be seen that although each pixel point can be randomly generated, the relative position relationship of each pixel point included in different target area ranges of the same processing task is consistent, that is, in any target area range, there are pixel points a to pixel point f, and the same pixel point in different target area ranges has the same position in the target area range. Since the implementation logic used in the process of obtaining pixel pairs for different key points is the same, the feature information of each key point obtained is also obtained based on the same consideration dimension, so that when the key point matching pair is determined based on the key point feature information corresponding to the key point, the obtained result can have a higher accuracy.

[0070] In specific implementation, when the shape of the target area is other regular shapes, for example, the target area can be a square area, a rectangular area, etc. When the target area is a square area, a square area with the key point as any vertex and a preset length as the side length can be determined as the target area range; or, a square area with the key point as the center point (i.e., the intersection of two diagonals in the square area) and a preset length as the diagonal length can be determined as the target area range; the subsequent process of generating key point feature information corresponding to the key point can refer to the process of step B2 to step B4, which will not be repeated here.

[0071] Here, multiple pixel pairs are determined within a target area with the key point as the center and a preset length as the radius, and the key point feature information corresponding to the key point is determined by using the pixel deviation information between the first pixel and the second pixel in the pixel pair, that is, the key point feature information includes the pixel deviation information of multiple pixel pairs around the key point, and the pixel distribution of the pixels around the key point is used to more accurately characterize the characteristics of the key point; so that the key point feature information can be used later to more accurately splice multiple frames of images to be processed.

[0072] For S104, based on the key point feature information corresponding to the key points in the multiple frames of images to be processed, the multiple frames of images to be processed are stitched together to generate a stitched image:

[0073] Since the key point feature information corresponding to the key point is determined based on the pixel deviation information of multiple pixel pairs within the target area corresponding to the key point, if the key point feature information corresponding to the key point 1 in the image to be processed in the adjacent images to be processed is 1011101011, the key point feature information corresponding to the key point 2 in the image to be processed is 1011101000, and the key point feature information corresponding to the key point 3 in the image to be processed is 1011100010; the difference between the key point feature information corresponding to the key point 1 and the key point feature information corresponding to the key point 2 is 3 (1011101011-1011101000), the key point feature information corresponding to the key point 1 is 1011101011-1011101000. The difference between the key point feature information corresponding to key point 3 is 9 (1011101011-1011100010). Since the deviation between the key point feature information of key point 1 and key point 2 is small, it means that the feature distribution within the target area corresponding to key point 1 is similar to the feature distribution within the target area corresponding to key point 2, and the possibility that key point 1 and key point 2 correspond to the same object to be identified is high, that is, it can be determined that the similarity of the object to be identified indicated by the key point feature information corresponding to key point 1 and the key point feature information corresponding to key point 2 is high; on the contrary, the similarity of the object to be identified indicated by the key point feature information corresponding to key point 1 and the key point feature information corresponding to key point 3 is low.

[0074] Therefore, the images can be spliced ​​based on the key point feature information corresponding to each key point in the adjacent images to be processed, that is, the overlapping areas corresponding to the adjacent images to be processed can be determined, and then the images to be processed can be deduplicated and spliced ​​to generate a spliced ​​image.

[0075] Taking into account the possibility of tilted scanning during the scanning process, which may cause misalignment of the objects to be identified in the generated stitched image, the adjacent images to be processed can be corrected and aligned based on the key point position information of each key point in the adjacent images to be processed, and then the corrected images to be processed can be registered, and the overlapping areas corresponding to the adjacent images to be processed can be determined, so that each frame of the images to be processed can be deduplicated and stitched to generate a stitched image.

[0076] In a possible implementation, based on key point feature information corresponding to key points in multiple frames of images to be processed, multiple frames of images to be processed are stitched together to generate a stitched image, including:

[0077] Step C1, for each group of adjacent images to be processed in multiple frames of the object to be processed, based on the key point feature information corresponding to each key point of the first image to be processed in the adjacent images to be processed and the key point feature information corresponding to each key point of the second image to be processed, determine the overlapping area information corresponding to the adjacent images to be processed.

[0078] Step C2: based on the obtained overlapping area information, multiple frames of images to be processed are stitched together to generate a stitched image.

[0079] Since the key point feature information corresponding to the key point is determined based on the pixel deviation information of multiple pixel pairs within the target area corresponding to the key point, if the similarity between the key point feature information corresponding to two key points is high, the possibility that the objects to be identified within the target area corresponding to the two key points are the same is high. Therefore, for each group of adjacent images to be processed in multiple frames of objects to be processed, the overlapping area information corresponding to the adjacent images to be processed can be determined based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed.

[0080] Then, the overlapping area for stitching can be selected first; for example, the overlapping area for stitching can be selected from the image area indicated by the overlapping area information corresponding to the first image to be processed; or, the image area indicated by the corresponding overlapping area information in the second image to be processed can be selected; or, the image area indicated by the overlapping area information corresponding to the first image to be processed can be regarded as the first image area, and the image area indicated by the corresponding overlapping area information in the second image to be processed can be regarded as the second image area, and the pixel point information of the pixel points at the same position (i.e., the same relative position) of the first image area and the second image area can be averaged to obtain the mean value corresponding to each position in the overlapping area, that is, the overlapping area can be obtained; further, based on the overlapping area information corresponding to multiple groups of adjacent images to be processed, multiple frames of images to be processed can be stitched to generate a stitched image, see Figure 5 shown.

[0081] Here, based on the key point feature information corresponding to each key point of the first image to be processed in each group of adjacent images to be processed, and the key point feature information corresponding to each key point of the second image to be processed, the overlapping area information corresponding to each group of adjacent images to be processed can be determined more accurately, thereby alleviating the interference of blank areas where there are no objects to be identified on image registration, and improving the accuracy of image registration to be processed. Based on the overlapping area information corresponding to multiple groups of adjacent images to be processed, multiple frames of images to be processed can be spliced ​​more accurately, so that the generated spliced ​​image is more accurate.

[0082] In a possible implementation, based on key point feature information corresponding to each key point of a first image to be processed and key point feature information corresponding to each key point of a second image to be processed, the overlapping area information corresponding to adjacent images to be processed is determined, including:

[0083] Step D1, based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, determine the key point matching pairs corresponding to adjacent images to be processed; wherein the key point matching pairs include the first key point on the first image to be processed and the second key point on the second image to be processed.

[0084] Step D2, based on the key point position information of the first key point in the key point matching pair corresponding to the adjacent images to be processed and the key point position information of the second key point, determine the overlapping area information corresponding to the adjacent images to be processed.

[0085] Since the key point feature information corresponding to the key point is determined based on the pixel deviation information of multiple pixel pairs within the target area corresponding to the key point, if the objects to be identified within the target area corresponding to the two key points are consistent, the similarity between the key point feature information corresponding to the two key points is high. Therefore, based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, the key point matching pairs corresponding to the adjacent images to be processed can be determined; wherein the key point matching pairs include the first key point on the first image to be processed and the second key point on the second image to be processed.

[0086] For example, for each key point 1 in the first image to be processed, a key point 2 having a key point feature similar to that of the key point 1 can be determined from the multiple key points 2 included in the second image to be processed, and the key point 1 and the determined key point 2 form a key point matching pair. If there is no key point 2 having a key point feature similar to that of the key point 1 among the multiple key points 2 included in the second image to be processed, it is determined that there is no matching key point 2 for the key point 1.

[0087] Alternatively, for each key point 2 in the second image to be processed, a key point 1 having a key point feature similar to that of the key point 2 may be determined from the multiple key points 1 included in the first image to be processed, and the key point 2 and the determined key point 1 may form a key point matching pair.

[0088] Then, the overlapping region information corresponding to the adjacent images to be processed can be determined based on the key point position information of the first key point in the key point matching pair corresponding to the adjacent images to be processed and the key point position information of the second key point. For example, the region corresponding to the key point matching pair can be determined as the overlapping region.

[0089] Here, if the objects to be identified within the target area corresponding to the two key points are consistent, the similarity between the key point feature information corresponding to the two key points is relatively high. Therefore, based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, the key point matching pairs corresponding to the adjacent images to be processed can be determined more accurately; and then based on the key point position information of the first key point in the key point matching pairs corresponding to the adjacent images to be processed and the key point position information of the second key point, the overlapping area information corresponding to the adjacent images to be processed can be determined more accurately.

[0090] In a possible implementation, step D1, based on key point feature information corresponding to each key point of the first image to be processed and key point feature information corresponding to each key point of the second image to be processed, determines key point matching pairs corresponding to adjacent images to be processed, including:

[0091] Step D11, for each key point of the first image to be processed, based on the key point feature information corresponding to the key point and the key point feature information corresponding to each key point of the second image to be processed, determine the similarity information between the key point and each key point of the second image to be processed.

[0092] Step D12: determining key point matching pairs corresponding to adjacent images to be processed based on similarity information between each key point of the first image to be processed and each key point of the second image to be processed.

[0093] During implementation, for each key point of the first image to be processed, the similarity information between the key point and each key point of the second image to be processed can be determined based on the key point feature information corresponding to the key point and the key point feature information corresponding to each key point of the second image to be processed.

[0094] For example, the Hamming distance between the key point feature information corresponding to the two key points can be determined as the similarity information between the two key points; or, the feature information difference between the key point feature information corresponding to the two key points can be determined, and the feature information difference can be determined as the similarity information between the two key points; wherein, the smaller the similarity information between the two key points, the higher the similarity between the two key points.

[0095] Furthermore, for the key point 1 of the first image to be processed, the key point 2 that matches the key point 1 can be determined based on the similarity information between the key point 1 of the first image to be processed and each key point 2 of the second image to be processed. When there is a key point 2 matching the key point 1, the key point 1 and the key point 2 constitute a key point matching pair. Then, the key point matching pairs corresponding to the adjacent images to be processed can be obtained.

[0096] Exemplarily, for key point 1 of the first image to be processed, key point 2 corresponding to the minimum similarity information may be determined as a key point that matches the key point 1. Alternatively, when the minimum similarity information is less than a preset threshold, key point 2 corresponding to the minimum similarity information may be determined as a key point that matches the key point 1; when the minimum similarity information is greater than or equal to the preset threshold, it is determined that key point 2 and key point 1 cannot form a key point matching pair.

[0097] For example, if the key point feature information corresponding to the key point 0 in the first image to be processed is 1011010, and there are key points 1 to 5 in the second image to be processed, the key point feature information of key point 1 is 1000110, the key point feature information of key point 2 is 1010110, the key point feature information of key point 3 is 1100100, the key point feature information of key point 4 is 0110101, and the key point feature information of key point 5 is 0110110; then the key point feature information corresponding to key point 0 and the key point feature information corresponding to key point 1 are The Hamming distance between the information is 5, that is, the similarity information between key point 0 and key point 1 is 5; the similarity information between key point 0 and key point 2 is 2; the similarity information between key point 0 and key point 3 is 5, the similarity information between key point 0 and key point 4 is 6, and the similarity information between key point 0 and key point 5 is 4; it can be determined that the similarity between key point 0 in the first image to be processed and key point 2 in the second image to be processed is high, that is, key point 0 in the first image to be processed and key point 2 in the second image to be processed constitute a key point matching pair.

[0098] Here, by utilizing the Hamming distance between the key point feature information corresponding to two key points, or the feature information difference, the similarity information between each key point of the first image to be processed and each key point of the second image to be processed can be determined more flexibly; and based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed, the key point matching pairs corresponding to adjacent images to be processed can be determined more accurately, so that the overlapping area information corresponding to the adjacent images to be processed can be determined subsequently based on the key point position information corresponding to the key points in the key point matching pairs corresponding to the adjacent images to be processed.

[0099] In a possible implementation, step D12, based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed, determines the key point matching pairs corresponding to adjacent images to be processed, including:

[0100] Step D121, for each key point to be processed in the first image to be processed, determine the first similarity information and the second similarity information from the similarity information between the key point to be processed and each key point of the second image to be processed in order from small to large; wherein the first similarity information is smaller than the second similarity information.

[0101] Step D122: Determine a similarity threshold corresponding to the key point to be processed based on the second similarity information.

[0102] Step D123, when the first similarity information is less than the similarity threshold, determine the key point corresponding to the first similarity information in the second image to be processed, and form a key point matching pair with the key point to be processed.

[0103] During implementation, for each key point to be processed in the first image to be processed, the first similarity information and the second similarity information can be determined from the similarity information between the key point to be processed and each key point of the second image to be processed in order from small to large; wherein the first similarity information is smaller than the second similarity information.

[0104] The first similarity information is the first similarity information in the sequence, that is, the minimum similarity information; the second similarity information is the second similarity information in the sequence, that is, the second minimum similarity information. Based on the second similarity information, the similarity threshold corresponding to the key point to be processed can be determined; illustratively, the product of the second similarity information and the preset coefficient can be determined as the similarity threshold corresponding to the key point to be processed; wherein the preset coefficient can be determined according to actual needs, for example, it can be 0.7, 0.8, etc.

[0105] When the first similarity information is less than the similarity threshold, it can be determined that the key points corresponding to the first similarity information in the second image to be processed constitute a key point matching pair with the key points to be processed; when the first similarity information is greater than or equal to the similarity threshold, it can be determined that each key point in the second image to be processed does not meet the conditions for constituting a key point matching pair with the key points to be processed, that is, the key points to be processed cannot constitute a key point matching pair.

[0106] Exemplarily, if the second image to be processed has key points 1 to 5, the key point to be processed in the first image to be processed is key point 0, and the similarity information 1 between key point 0 and key point 1 is 5; the similarity information 2 between key point 0 and key point 2 is 2; the similarity information 3 between key point 0 and key point 3 is 5, the similarity information 4 between key point 0 and key point 4 is 6, and the similarity information 5 between key point 0 and key point 5 is 4; in order from small to large, the first similarity information is determined to be similarity information 2, and the second similarity information is determined to be similarity information 5; if the preset coefficient is 0.8, the similarity threshold is 3.2 (0.8×4), and it can be determined that the key point 2 corresponding to the similarity information 2 in the second image to be processed constitutes a key point matching pair with the key point 0.

[0107] Here, based on the second similarity information corresponding to each key point to be processed in the first image to be processed, the similarity threshold corresponding to the key point to be processed can be determined more flexibly and accurately; when the first similarity information corresponding to the key point to be processed is less than the similarity threshold, it is determined that the key point corresponding to the first similarity information in the second image to be processed has a high similarity with the key point to be processed, and can form a key point matching pair, and the determination of the key point matching pair is more accurate.

[0108] In a possible implementation, step D2, based on key point position information of a first key point in a key point matching pair corresponding to adjacent images to be processed and key point position information of a second key point, determines overlapping area information corresponding to adjacent images to be processed, including:

[0109] Step D21, for each key point matching pair in the adjacent to-be-processed images, based on the key point position information of the first key point and the key point position information of the second key point in the key point matching pair, determine the intermediate movement vector corresponding to the key point matching pair.

[0110] Step D22: determining the motion vectors corresponding to the adjacent images to be processed based on the intermediate motion vectors corresponding to the key point matching pairs.

[0111] Step D23, determining a moving distance between a first image to be processed and a second image to be processed in the adjacent images to be processed based on the moving vectors corresponding to the adjacent images to be processed.

[0112] Step D24, determining overlapping area information corresponding to adjacent images to be processed based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed.

[0113] During implementation, for each key point matching pair in adjacent images to be processed, the intermediate motion vector corresponding to the key point matching pair can be determined based on the key point position information of the first key point and the key point position information of the second key point in the key point matching pair; for example, the key point position information of the first key point is (x1, y1), and the key point position information of the second key point is (x2, y2), then the intermediate motion vector corresponding to the key point matching pair is (x2-x1, y2-y1); then the motion vector corresponding to the adjacent images to be processed can be determined based on the intermediate motion vectors corresponding to each key point matching pair; for example, the average value of the intermediate motion vectors corresponding to each key point matching pair can be determined as the motion vector corresponding to the adjacent images to be processed; or, the median of the intermediate motion vectors corresponding to each key point matching pair can be determined as the motion vector corresponding to the adjacent images to be processed, and so on.

[0114] Then, based on the motion vector d(x, y) corresponding to the adjacent images to be processed, the motion distance between the first image to be processed and the second image to be processed in the adjacent images to be processed can be determined, for example, according to the formula: Determine the moving distance; further, based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, the overlapping area information corresponding to the adjacent images to be processed can be determined; illustratively, the difference between the moving distance between the first image to be processed and the second image to be processed and the distance between the adjacent images to be processed can be determined as the width of the overlapping area information corresponding to the adjacent images to be processed; the height of the preset image size is determined as the height of the overlapping area information corresponding to the adjacent images to be processed.

[0115] Here, after determining the intermediate motion vector corresponding to each key point matching pair in adjacent images to be processed, the motion vector corresponding to the adjacent images to be processed can be determined more accurately based on the intermediate motion vectors corresponding to each key point matching pair; and based on the motion vector corresponding to the adjacent images to be processed, the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed is determined; so that the overlapping area information corresponding to the adjacent images to be processed determined based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed is more accurate, so that the stitching effect of the stitched image generated based on the overlapping area information corresponding to multiple groups of adjacent images to be processed is better.

[0116] In a possible implementation manner, after determining the motion vectors corresponding to the adjacent to-be-processed images based on the intermediate motion vectors corresponding to the key point matching pairs, the method further includes:

[0117] Step E1, generating rotation angles of the objects to be identified corresponding to multiple frames of images to be processed based on motion vectors corresponding to adjacent images to be processed.

[0118] Step E2, generating a rotation matrix corresponding to multiple frames of images to be processed based on the rotation angle and the set rotation reference direction.

[0119] Step E3, using the rotation matrix, correcting the first image to be processed and the second image to be processed in the adjacent images to be processed to generate the corrected first image to be processed and the corrected second image to be processed; and using the rotation matrix, correcting the motion vectors corresponding to the adjacent images to be processed to generate the corrected motion vectors.

[0120] Determining a moving distance between a first image to be processed and a second image to be processed in the adjacent images to be processed based on a moving vector corresponding to the adjacent images to be processed, and determining overlapping area information corresponding to the adjacent images to be processed based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, including:

[0121] Step E4, based on the corrected motion vectors corresponding to the adjacent images to be processed, determining the motion distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed.

[0122] Step E5, determining the overlapping area information corresponding to the adjacent images to be processed based on the moving distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed.

[0123] During implementation, the rotation angles of the objects to be identified corresponding to the multiple frames of images to be processed can be generated based on the motion vectors corresponding to the adjacent images to be processed; specifically, the rotation angles can be generated using trigonometric functions; for example, if the motion vector is d(x, y), the rotation angle θ can be arctan(y / x); then, based on the rotation angle θ and the set rotation reference direction (for example, the horizontal direction), the rotation matrix corresponding to the multiple frames of images to be processed can be generated; for example, the rotation matrix can be as follows:

[0124]

[0125] Furthermore, the rotation matrix can be used to correct the first image to be processed and the second image to be processed in the adjacent images to be processed, so as to generate the corrected first image to be processed and the corrected second image to be processed. Specifically, the pixel position information of each pixel point in the first image to be processed and the second image to be processed can be multiplied by the rotation matrix to obtain the corrected pixel position information; each corrected pixel position information can constitute the corrected first image to be processed and the corrected second image to be processed. And the rotation matrix and the motion vector corresponding to each group of adjacent images to be processed can be multiplied to obtain the corrected motion vector.

[0126] During implementation, the moving distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed can be determined based on the corrected moving vectors corresponding to the adjacent images to be processed; and the overlapping area information corresponding to the adjacent images to be processed can be determined based on the moving distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed. The specific process can be referred to in the above steps D23 and D24, which will not be described in detail here.

[0127] Taking into account the situation that the scanning process causes tilted scanning due to the shaking of the scanning device, resulting in the corresponding bends to be identified in the scanned image, and the objects to be identified included in the overlapping areas of the first image to be processed and the second image to be processed are not aligned, in order to alleviate the above situation, the rotation angles of the objects to be identified corresponding to the multiple frames of images to be processed can be generated more accurately based on the motion vectors corresponding to the adjacent images to be processed; and the rotation matrix corresponding to the multiple frames of images to be processed can be generated based on the rotation angle and the set rotation reference direction; then, the first image to be processed and the second image to be processed in the adjacent images to be processed can be corrected by using the rotation matrix to generate the corrected first image to be processed and the corrected second image to be processed; and the rotation matrix can be used to correct the motion vectors corresponding to the adjacent images to be processed to generate the corrected motion vectors; subsequently, the overlapping area information corresponding to the adjacent images to be processed can be determined based on the corrected motion vectors corresponding to the adjacent images to be processed, and then the multiple frames of corrected images to be processed can be spliced ​​using the overlapping area information, so that the objects to be identified in the generated spliced ​​image are easier to identify.

[0128] Considering that the user may shake when scanning text content with a scanning device, so that the object to be identified in the spliced ​​image may still be unclear, the object to be identified in the spliced ​​image can be detected and adjusted. At the same time, multiple lines of text content will be obtained during the scanning process, but the text content in the center line of the multiple lines of text content is the current scanning content of the scanning device, that is, the image area corresponding to the middle height of the spliced ​​image, so the target image can be intercepted from the spliced ​​image.

[0129] In a possible implementation manner, after generating the spliced ​​image, the method further includes:

[0130] Step F1, performing object detection on the stitched image to obtain detection frame information corresponding to at least one object to be identified in the stitched image.

[0131] Step F2, based on the detection frame information corresponding to at least one object to be identified in the stitched image, determine a target image including target information from the stitched image; wherein the target information includes a plurality of objects to be identified that match the continuous scanning trajectory.

[0132] During implementation, after obtaining the stitched image, an object detection algorithm (such as CTPN algorithm, EAST algorithm, etc.) can be used to perform object detection on the stitched image to obtain detection frame information corresponding to at least one object to be identified in the stitched image; wherein the width of the detection frame can be set in advance, that is, the width of the detection frame corresponding to the object to be identified is consistent, and the height of the detection frame corresponding to the object to be identified is obtained by the object detection algorithm. Then, based on the detection frame information corresponding to at least one object to be identified in the stitched image, a target image including target information can be determined from the stitched image; the target information includes multiple objects to be identified that match the continuous scanning trajectory, see Figure 6 As shown, the multiple objects to be identified in the detection box are target information.

[0133] Here, by performing object detection on the stitched image, detection frame information corresponding to at least one object to be identified in the stitched image can be obtained, and based on the detection frame information corresponding to at least one object to be identified in the stitched image, a target image including target information can be more accurately determined from the stitched image.

[0134] In a possible implementation, step F2, determining a target image including target information from the stitched image based on detection frame information corresponding to at least one object to be identified in the stitched image, includes:

[0135] Step F21, determining at least one object to be identified in each vertical interval based on the detection frame information corresponding to at least one object to be identified in the stitched image;

[0136] Step F22, determining a target vertical interval based on the interval ranges corresponding to the respective vertical intervals; wherein at least one to-be-identified object located within the target vertical interval is target information;

[0137] Step F23 , obtaining a target image from the stitched image based on the detection frame information corresponding to at least one object to be identified within the target vertical interval.

[0138] During implementation, at least one object to be identified in each vertical interval can be determined based on the detection frame information corresponding to at least one object to be identified in the stitched image, that is, at least one object to be identified in the same row can be determined; for example, the first row is the one in which the vertical coordinate indicated by the detection frame information is in the interval of 0 to -18 pixels, the second row is the one in which the vertical coordinate indicated by the detection frame information is in the interval of -20 to -38 pixels, and the third row is the one in which the vertical coordinate indicated by the detection frame information is in the interval of -40 to -58 pixels; and then the target vertical interval can be determined based on the interval range corresponding to each vertical interval, that is, the target vertical interval is determined to be the one in the middle of the interval range corresponding to the vertical interval, and the target vertical interval is -20 to -38 pixels; wherein, at least one object to be identified in the target vertical interval is the target information; and further, the target image can be acquired from the stitched image based on the detection frame information corresponding to at least one object to be identified in the target vertical interval.

[0139] In specific implementation, step F23, based on the detection frame information corresponding to at least one object to be identified in the target vertical interval, obtains the target image from the stitched image, including:

[0140] Step F231, determining an average height of the detection frame in the target vertical interval based on the detection frame information corresponding to at least one object to be identified in the target vertical interval;

[0141] Step F232, adjusting the detection frame information corresponding to at least one to-be-recognized object in the target vertical interval based on the average height, and generating adjusted detection frame information;

[0142] Step F233, based on the adjusted detection frame information, intercepting at least one partial image corresponding to the object to be identified from the stitched image;

[0143] Step F234: Generate a target image by stitching based on at least one partial image corresponding to the object to be identified.

[0144] During implementation, the average height of the detection frame in the target vertical interval can be determined based on the detection frame information corresponding to at least one object to be identified in the target vertical interval; and the detection frame information corresponding to at least one object to be identified in the target vertical interval can be adjusted based on the average height to generate adjusted detection frame information; that is, the sizes of the adjusted detection frames are the same; for example, the detection frame whose height is less than the average height can be enlarged to obtain the adjusted detection frame and the adjusted detection frame information; the detection frame whose height is greater than the average height can be reduced to obtain the adjusted detection frame and the adjusted detection frame information.

[0145] Then, based on the adjusted detection frame information, at least one partial image corresponding to the object to be identified can be intercepted from the stitched image; that is, the image area indicated by the adjusted detection frame information can be intercepted from the stitched image to obtain at least one partial image corresponding to the object to be identified; further, the partial images corresponding to the at least one object to be identified can be stitched to generate a target image, see Figure 7 shown.

[0146] Alternatively, the initial target image can be first obtained from the stitched image based on the interval range corresponding to the target vertical interval; for example, if the target vertical interval is -20 to -38 pixels, then the image area located between -20 and -38 pixels is obtained from the stitched image as the initial target image; then, based on the detection frame information corresponding to the multiple objects to be identified in the target vertical interval, the positions of the multiple objects to be identified in the initial target image can be adjusted to generate multiple adjusted objects to be identified located at the same horizontal position, and an adjusted image including the multiple adjusted objects to be identified; further, the adjusted image including the multiple adjusted objects to be identified can be determined as the target image.

[0147] In the above method, the detection frame information corresponding to at least one object to be identified within the target vertical interval is adjusted so that the height of the adjusted detection frame is consistent, and based on the adjusted detection frame information, at least one local image corresponding to the object to be identified is captured from the stitched image so that the size of each local image corresponding to the object to be identified is the same; then, the local image corresponding to at least one object to be identified is stitched to generate a target image, in which the target information is clearer, the display is smoother, and it is easy to identify, which alleviates the problem of tilt of the object to be identified captured by the camera during the scanning process and improves the display effect of the target image.

[0148] Here, after obtaining the stitched image, based on the detection frame information corresponding to at least one object to be identified in the stitched image, at least one object to be identified in each vertical interval is determined; and based on the interval ranges corresponding to each vertical interval, the target vertical interval in the center of the interval range is determined, and the target vertical interval is the scanning interval corresponding to the current scanning process; furthermore, based on the detection frame information corresponding to at least one object to be identified in the target vertical interval, the target image can be obtained from the stitched image more accurately.

[0149] In a possible implementation, after determining the target image including the target information from the stitched image, the method further includes: using a text recognition algorithm to recognize the target information in the target image to determine the text information included in the target image.

[0150] Output text information in at least one of the following ways:

[0151] The first method is to use a voice playback device to play the text information and / or to display the text information through a display device.

[0152] The second method is to convert the language of the text information to obtain the converted text information, and use a voice playback device to play the converted text information, and / or display the converted text information through a display device.

[0153] During implementation, after determining the target image including the target information from the stitched image, a text recognition algorithm (Optical Character Recognition, OCR) can be used to recognize the target information in the target image to determine the text information included in the target image; and the text information can be played using a voice playback device; or the text information can be displayed through a display device; or the text information can be displayed through a display device and the text information can be played using a voice playback device at the same time.

[0154] The text information can also be converted into different languages ​​to obtain converted text information, for example, Chinese characters can be converted into English words, English words can be converted into French words, etc. The converted text information can then be played using a voice playback device, and / or displayed using a display device.

[0155] Then, the text information can be played by using a voice playing device, and the language of the text information can be converted to obtain converted text information, and the converted text information can be played by using a voice playing device.

[0156] Here, since the target information in the target image obtained by any of the above implementations is clearer and more accurate, the target information in the target image is recognized by using a text recognition algorithm, which can simply and accurately recognize the text information included in the target image.

[0157] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0158] Based on the same concept, the present disclosure also provides an image processing device, see Figure 8 As shown, it is a schematic diagram of the architecture of the image processing device provided by the embodiment of the present disclosure, including an acquisition module 801, a screening module 802, a first determination module 803, and a first generation module 804. Specifically:

[0159] An acquisition module 801 is used to acquire multiple frames of images to be processed obtained by continuously scanning text content;

[0160] A screening module 802 is used for screening out key point position information corresponding to the object to be identified in each frame of the plurality of frames of the images to be processed from each pixel based on the pixel information of each pixel in the image to be processed;

[0161] A first determination module 803 is used to obtain multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs;

[0162] The first generating module 804 is configured to stitch the multiple frames of images to be processed based on key point feature information corresponding to each key point in each frame of the images to be processed to generate a stitched image.

[0163] In a possible implementation manner, the screening module 802, when screening out a key corresponding to an object to be identified in the image to be processed from each pixel based on pixel information of each pixel in the image to be processed, is configured to:

[0164] Taking each pixel point of at least part of the pixels of the to-be-processed image as a candidate key pixel point, and determining a pixel difference between pixel information of each boundary pixel point on a neighborhood boundary corresponding to the candidate key pixel point and pixel information of the candidate key pixel point;

[0165] Determine the boundary pixel points whose pixel difference is greater than the difference threshold as candidate boundary pixel points;

[0166] When the number of candidate boundary pixel points that appear continuously on the neighborhood boundary is greater than a preset number, the candidate key pixel points are determined as key points corresponding to the object to be identified.

[0167] In a possible implementation manner, when the target area range includes a circular area, the first determination module 803, when acquiring multiple groups of pixel pairs within the target area range corresponding to the key point and obtaining key point feature information corresponding to the key point based on the multiple groups of pixel pairs, is used to:

[0168] Determine the target area with the key point as the center and a preset length as the radius;

[0169] Determine, within the target area, a plurality of groups of pixel pairs corresponding to the key points;

[0170] For each pixel point pair in the plurality of pixel point pairs, determining pixel deviation information between pixel information of a first pixel point and pixel information of a second pixel point in the pixel point pair;

[0171] Based on the obtained pixel deviation information respectively corresponding to the multiple groups of pixel point pairs, key point feature information corresponding to the key point is generated.

[0172] In a possible implementation manner, the first generating module 804, when stitching the multiple frames of images to be processed based on the key point feature information corresponding to each key point in the multiple frames of images to be processed to generate a stitched image, is used to:

[0173] For each group of adjacent images to be processed in the multiple frames of images to be processed, based on the key point feature information corresponding to each key point of a first image to be processed in the adjacent images to be processed and the key point feature information corresponding to each key point of a second image to be processed, determining overlapping area information corresponding to the adjacent images to be processed;

[0174] Based on the obtained overlapping area information, the multiple frames of images to be processed are spliced ​​to generate a spliced ​​image.

[0175] In a possible implementation manner, the first generating module 804, when determining the overlapping area information corresponding to any adjacent image to be processed based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, is used to:

[0176] Based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, determining a key point matching pair corresponding to the adjacent images to be processed; wherein the key point matching pair includes a first key point on the first image to be processed and a second key point on the second image to be processed;

[0177] Based on the key point position information of the first key point in the key point matching pairs corresponding to the adjacent images to be processed and the key point position information of the second key point, the overlapping area information corresponding to the adjacent images to be processed is determined.

[0178] In a possible implementation manner, the first generating module 804, when determining the key point matching pairs corresponding to the adjacent images to be processed based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, is configured to:

[0179] For each key point of the first image to be processed, based on the key point feature information corresponding to the key point and the key point feature information corresponding to each key point of the second image to be processed, determining similarity information between the key point and each key point of the second image to be processed;

[0180] Based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed, a key point matching pair corresponding to the adjacent images to be processed is determined.

[0181] In a possible implementation manner, the first generating module 804, when determining the key point matching pairs corresponding to the adjacent images to be processed based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed, is configured to:

[0182] For each key point to be processed in the first image to be processed, determining first similarity information and second similarity information from similarity information between the key point to be processed and each key point of the second image to be processed in ascending order; wherein the first similarity information is smaller than the second similarity information;

[0183] Based on the second similarity information, determining a similarity threshold corresponding to the key point to be processed;

[0184] When the first similarity information is less than the similarity threshold, it is determined that a key point corresponding to the first similarity information in the second image to be processed forms a key point matching pair with the key point to be processed.

[0185] In a possible implementation manner, the first generating module 804, when determining the overlapping area information corresponding to the adjacent images to be processed based on the key point position information of the first key point in the key point matching pair corresponding to the adjacent images to be processed and the key point position information of the second key point, is used to:

[0186] For each of the key point matching pairs in the adjacent to-be-processed images, based on the key point position information of the first key point and the key point position information of the second key point in the key point matching pair, determine an intermediate movement vector corresponding to the key point matching pair;

[0187] Determine the motion vector corresponding to the adjacent image to be processed based on the intermediate motion vectors corresponding to each of the key point matching pairs;

[0188] Determining a moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed based on the moving vectors corresponding to the adjacent images to be processed;

[0189] Based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, overlapping area information corresponding to the adjacent images to be processed is determined.

[0190] In a possible implementation manner, the device further includes: a second generating module 805, wherein the second generating module 805, after determining the motion vector corresponding to the adjacent to-be-processed image based on the intermediate motion vectors corresponding to each of the key point matching pairs, is configured to:

[0191] Generate a rotation angle of the object to be identified corresponding to the plurality of frames of images to be processed based on the motion vectors corresponding to the adjacent images to be processed;

[0192] Based on the rotation angle and the set rotation reference direction, generating a rotation matrix corresponding to the multiple frames of images to be processed;

[0193] Using the rotation matrix, correcting the first image to be processed and the second image to be processed in the adjacent images to be processed to generate a corrected first image to be processed and a corrected second image to be processed; and

[0194] Using the rotation matrix, correcting the motion vectors corresponding to the adjacent images to be processed to generate corrected motion vectors;

[0195] The first generating module 804, when determining the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed based on the moving vectors corresponding to the adjacent images to be processed, and determining the overlapping area information corresponding to the adjacent images to be processed based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, is used to:

[0196] Determining a movement distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed based on the corrected movement vectors corresponding to the adjacent images to be processed;

[0197] Based on the moving distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed, overlapping area information corresponding to the adjacent images to be processed is determined.

[0198] In a possible implementation manner, the apparatus further includes: a second determining module 806, wherein the second determining module 806, after generating the stitched image, is configured to:

[0199] Performing object detection on the stitched image to obtain detection frame information corresponding to at least one of the objects to be identified in the stitched image;

[0200] Based on the detection frame information corresponding to at least one of the objects to be identified in the stitched image, a target image including target information is determined from the stitched image; wherein the target information includes a plurality of objects to be identified that match the continuous scanning trajectory.

[0201] In a possible implementation manner, the second determination module 806, when determining the target image including the target information from the stitched image based on the detection frame information corresponding to at least one of the to-be-recognized objects in the stitched image, is configured to:

[0202] Determine at least one of the objects to be identified located in each vertical interval based on the detection frame information corresponding to at least one of the objects to be identified in the stitched image;

[0203] Determine a target vertical interval based on the interval ranges corresponding to each vertical interval; wherein the at least one object to be identified within the target vertical interval is the target information;

[0204] The target image is acquired from the stitched image based on the detection frame information corresponding to at least one to-be-recognized object within the target vertical interval.

[0205] In a possible implementation manner, the second determination module 806, when acquiring the target image from the stitched image based on the detection frame information corresponding to at least one object to be identified in the target vertical interval, is configured to:

[0206] Determining an average height of the detection frame in the target vertical interval based on the detection frame information corresponding to the at least one object to be identified in the target vertical interval;

[0207] Based on the average height, adjusting the detection frame information corresponding to the at least one to-be-recognized object in the target vertical interval to generate adjusted detection frame information;

[0208] Based on the adjusted detection frame information, intercepting a partial image corresponding to the at least one object to be identified from the stitched image;

[0209] The target image is generated by stitching based on the local image corresponding to the at least one object to be identified.

[0210] In a possible implementation manner, the apparatus further includes: an identification module 807, wherein the identification module 807, after determining the target image including the target information from the stitched image, is configured to:

[0211] Using a text recognition algorithm, the target information in the target image is recognized to determine the text information included in the target image;

[0212] Output the text information in at least one of the following ways:

[0213] Playing the text information using a voice playback device, and / or displaying the text information using a display device;

[0214] The text information is converted into a language to obtain converted text information, and the converted text information is played by a voice playback device, and / or displayed by the display device.

[0215] In some embodiments, the functions or templates contained in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0216] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig. 9 As shown, it is a schematic diagram of the structure of the electronic device 900 provided in the embodiment of the present disclosure, including a processor 901, a memory 902, and a bus 903. Among them, the memory 902 is used to store execution instructions, including a memory 9021 and an external memory 9022; the memory 9021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 901, and the data exchanged with the external memory 9022 such as a hard disk. The processor 901 exchanges data with the external memory 9022 through the memory 9021. When the electronic device 900 is running, the processor 901 communicates with the memory 902 through the bus 903, so that the processor 901 executes the following instructions:

[0217] Acquire multiple frames of images to be processed after continuously scanning the text content;

[0218] For each frame of the image to be processed in the multiple frames of the image to be processed, based on pixel point information of each pixel point in the image to be processed, a key point corresponding to the object to be identified in the image to be processed is screened out from each pixel point;

[0219] Acquire multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs;

[0220] Based on the key point feature information corresponding to the key points in the multiple frames of images to be processed, the multiple frames of images to be processed are spliced ​​to generate a spliced ​​image.

[0221] Among them, the specific processing flow of the processor 901 can refer to the records of the above-mentioned method embodiment, which will not be repeated here.

[0222] In addition, the embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image processing method described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0223] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the image processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0224] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0225] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0226] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0228] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0229] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An image processing method, It is characterized in that include: Acquire multiple frames of images to be processed after continuously scanning the text content; For each frame of the image to be processed in the multiple frames of the image to be processed, based on pixel point information of each pixel point in the image to be processed, a key point corresponding to the object to be identified in the image to be processed is screened out from each pixel point; Acquire multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs; Based on key point feature information corresponding to key points in the multiple frames of images to be processed, stitching the multiple frames of images to be processed to generate a stitched image; The method of screening out key points corresponding to the object to be identified in the image to be processed from the pixel points based on the pixel information of each pixel point in the image to be processed includes: taking each pixel point in at least part of the pixel points in the image to be processed as a candidate key pixel point, and determining the pixel difference between the pixel information of each boundary pixel point on the neighborhood boundary corresponding to the candidate key pixel point and the pixel information of the candidate key pixel point; determining the boundary pixel point whose pixel difference is greater than a difference threshold as a candidate boundary pixel point; and determining the candidate key pixel point as the key point corresponding to the object to be identified when the number of candidate boundary pixel points appearing continuously on the neighborhood boundary is greater than a preset number; The method of stitching the multiple frames of images to be processed based on the key point feature information corresponding to the key points in the multiple frames of images to be processed to generate a stitched image includes: correcting and aligning the adjacent images to be processed based on the key point position information of each key point in the adjacent images to be processed, registering the corrected images to be processed, determining the overlapping areas corresponding to the adjacent images to be processed, deduplicating and stitching the frames of images to be processed to generate a stitched image.

2. The method according to claim 1, It is characterized in that In the case where the target area includes a circular area, acquiring a plurality of groups of pixel pairs within the target area corresponding to the key point, and obtaining key point feature information corresponding to the key point based on the plurality of groups of pixel pairs, includes: Determine the target area with the key point as the center and a preset length as the radius; Determine, within the target area, a plurality of groups of pixel pairs corresponding to the key points; For each pixel point pair in the plurality of pixel point pairs, determining pixel deviation information between pixel information of a first pixel point and pixel information of a second pixel point in the pixel point pair; Based on the obtained pixel deviation information respectively corresponding to the multiple groups of pixel point pairs, key point feature information corresponding to the key point is generated.

3. The method according to claim 1 or 2, It is characterized in that The step of stitching the multiple frames of images to be processed based on the key point feature information corresponding to the key points in the multiple frames of images to be processed to generate a stitched image includes: For each group of adjacent images to be processed in the multiple frames of images to be processed, based on the key point feature information corresponding to each key point of a first image to be processed in the adjacent images to be processed and the key point feature information corresponding to each key point of a second image to be processed, determining overlapping area information corresponding to the adjacent images to be processed; Based on the obtained overlapping area information, the multiple frames of images to be processed are spliced ​​to generate a spliced ​​image.

4. The method according to claim 3, It is characterized in that The determining, based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, the overlapping area information corresponding to the adjacent images to be processed includes: Based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed, determining a key point matching pair corresponding to the adjacent images to be processed; wherein the key point matching pair includes a first key point on the first image to be processed and a second key point on the second image to be processed; Based on the key point position information of the first key point in the key point matching pairs corresponding to the adjacent images to be processed and the key point position information of the second key point, the overlapping area information corresponding to the adjacent images to be processed is determined.

5. The method according to claim 4, It is characterized in that The determining the key point matching pairs corresponding to the adjacent images to be processed based on the key point feature information corresponding to each key point of the first image to be processed and the key point feature information corresponding to each key point of the second image to be processed includes: For each key point of the first image to be processed, based on the key point feature information corresponding to the key point and the key point feature information corresponding to each key point of the second image to be processed, determining similarity information between the key point and each key point of the second image to be processed; Based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed, a key point matching pair corresponding to the adjacent images to be processed is determined.

6. The method according to claim 5, It is characterized in that The determining key point matching pairs corresponding to the adjacent images to be processed based on the similarity information between each key point of the first image to be processed and each key point of the second image to be processed respectively includes: For each key point to be processed in the first image to be processed, determining first similarity information and second similarity information from similarity information between the key point to be processed and each key point of the second image to be processed in ascending order; wherein the first similarity information is smaller than the second similarity information; Based on the second similarity information, determining a similarity threshold corresponding to the key point to be processed; When the first similarity information is less than the similarity threshold, it is determined that a key point corresponding to the first similarity information in the second image to be processed forms a key point matching pair with the key point to be processed.

7. The method according to any one of claims 4 to 6, It is characterized in that The determining the overlapping area information corresponding to the adjacent images to be processed based on the key point position information of the first key point in the key point matching pair corresponding to the adjacent images to be processed and the key point position information of the second key point includes: For each of the key point matching pairs in the adjacent to-be-processed images, based on the key point position information of the first key point and the key point position information of the second key point in the key point matching pair, determine an intermediate movement vector corresponding to the key point matching pair; Determine the motion vector corresponding to the adjacent image to be processed based on the intermediate motion vectors corresponding to each of the key point matching pairs; Determining a moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed based on the moving vectors corresponding to the adjacent images to be processed; Based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, overlapping area information corresponding to the adjacent images to be processed is determined.

8. The method according to claim 7, It is characterized in that After determining the motion vectors corresponding to the adjacent images to be processed based on the intermediate motion vectors corresponding to the key point matching pairs, the method further includes: Generate a rotation angle of the object to be identified corresponding to the plurality of frames of images to be processed based on the motion vectors corresponding to the adjacent images to be processed; Based on the rotation angle and the set rotation reference direction, generating a rotation matrix corresponding to the multiple frames of images to be processed; Using the rotation matrix, correcting the first image to be processed and the second image to be processed in the adjacent images to be processed to generate a corrected first image to be processed and a corrected second image to be processed; and Using the rotation matrix, correcting the motion vectors corresponding to the adjacent images to be processed to generate corrected motion vectors; The determining, based on the motion vectors corresponding to the adjacent images to be processed, of the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, and the determining, based on the moving distance between the first image to be processed and the second image to be processed in the adjacent images to be processed, of the overlapping area information corresponding to the adjacent images to be processed, includes: Determining a movement distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed based on the corrected movement vectors corresponding to the adjacent images to be processed; Based on the moving distance between the corrected first image to be processed and the corrected second image to be processed in the adjacent images to be processed, overlapping area information corresponding to the adjacent images to be processed is determined.

9. The method according to any one of claims 1 to 8, It is characterized in that After the stitched image is generated, the method further includes: Performing object detection on the stitched image to obtain detection frame information corresponding to at least one of the objects to be identified in the stitched image; Based on the detection frame information corresponding to at least one of the objects to be identified in the stitched image, a target image including target information is determined from the stitched image; wherein the target information includes a plurality of objects to be identified that match the continuous scanning trajectory.

10. The method according to claim 9, It is characterized in that The step of determining a target image including target information from the stitched image based on the detection frame information corresponding to at least one of the objects to be identified in the stitched image comprises: Determine at least one of the objects to be identified located in each vertical interval based on the detection frame information corresponding to at least one of the objects to be identified in the stitched image; Determine a target vertical interval based on the interval ranges corresponding to each vertical interval; wherein the at least one object to be identified within the target vertical interval is the target information; The target image is acquired from the stitched image based on the detection frame information corresponding to at least one to-be-recognized object within the target vertical interval.

11. The method according to claim 10, It is characterized in that The acquiring the target image from the stitched image based on the detection frame information corresponding to at least one to-be-recognized object within the target vertical interval includes: Determining an average height of the detection frame in the target vertical interval based on the detection frame information corresponding to the at least one object to be identified in the target vertical interval; Based on the average height, adjusting the detection frame information corresponding to the at least one to-be-recognized object in the target vertical interval to generate adjusted detection frame information; Based on the adjusted detection frame information, intercepting a partial image corresponding to the at least one object to be identified from the stitched image; The target image is generated by stitching based on the local image corresponding to the at least one object to be identified.

12. The method according to any one of claims 9 to 11, It is characterized in that After determining the target image including the target information from the stitched images, the method further includes: Using a text recognition algorithm, the target information in the target image is recognized to determine the text information included in the target image; Output the text information in at least one of the following ways: Playing the text information using a voice playback device, and / or displaying the text information using a display device; The text information is converted into a language to obtain converted text information, and the converted text information is played by a voice playback device, and / or displayed by the display device.

13. An image processing device, It is characterized in that include: An acquisition module, used to acquire multiple frames of images to be processed obtained by continuously scanning the text content; A screening module, for screening out key points corresponding to the objects to be identified in each frame of the plurality of frames of the images to be processed, from each pixel based on the pixel information of each pixel in the image to be processed; A first determination module is used to obtain multiple groups of pixel pairs within the target area corresponding to the key point, and obtain key point feature information corresponding to the key point based on the multiple groups of pixel pairs; A first generating module, configured to splice the multiple frames of images to be processed based on key point feature information corresponding to key points in the multiple frames of images to be processed to generate a spliced ​​image; The screening module, when screening out key points corresponding to the object to be identified in the image to be processed from the pixels based on the pixel information of each pixel in the image to be processed, is used to: take each pixel in at least part of the pixels of the image to be processed as a candidate key pixel, and determine the pixel difference between the pixel information of each boundary pixel on the neighborhood boundary corresponding to the candidate key pixel and the pixel information of the candidate key pixel; Determine the boundary pixel points whose pixel difference is greater than the difference threshold as candidate boundary pixel points; when the number of candidate boundary pixel points that appear continuously on the neighborhood boundary is greater than a preset number, determine the candidate key pixel points as key points corresponding to the object to be identified; The first generating module, when stitching the multiple frames of images to be processed based on the key point feature information corresponding to the key points in the multiple frames of images to be processed to generate a stitched image, is used to: based on the key point position information of each key point in the adjacent images to be processed, correct and align the adjacent images to be processed, then align the corrected images to be processed, and determine the overlapping areas corresponding to the adjacent images to be processed, deduplicate and stitch each frame of the images to be processed to generate a stitched image.

14. An electronic device, It is characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method as described in any one of claims 1 to 12 are performed.

15. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 12 are executed.