Target image acquisition method and related equipment

Abstract

The present disclosure provides a method for acquiring a target image and related equipment. The method includes: acquiring multiple image frames about the target image frame by frame; wherein the image frame includes a partial image of the target image; a segmentation step: segmenting the current image frame based on the i-th preset size among n different preset sizes to obtain an i-th size image; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer; a matching step: performing image matching on the i-th size image with the previous image frame of the current image frame to obtain the i-th matching degree between the i-th size image and the previous image frame and the i-th offset of the i-th size image in the previous image frame; a fusion step: in response to the i-th matching degree being within a preset range, fusing the current image frame with the previous image frame based on the i-th offset.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method for acquiring a target image and related equipment. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Dictionary pens are a common learning tool for contemporary students. Their scan-and-check feature simplifies the process of looking up words and sentences for students, greatly improving learning efficiency. Existing dictionary pens use a camera installed on the tip of the pen to scan the content to be checked to obtain multiple frames of scanned images, and then match and splice the multiple frames of scanned images based on the feature points of each frame of the scanned image, and finally obtain a target image including the content to be checked. However, this method based on feature point matching and splicing has a relatively large amount of calculation, resulting in a large delay in obtaining the target image, which reduces the user experience. At the same time, due to the limitation of the field of view of the dictionary pen camera, it may not be possible to obtain enough feature points, resulting in matching failures or errors, resulting in poor results for the target image. Summary of the invention

[0004] In view of this, an improved method is highly needed to effectively improve the problem of large delay and poor effect in acquiring the target image.

[0005] An exemplary embodiment of the present disclosure provides a method for acquiring a target image, comprising:

[0006] Acquire multiple image frames about the target image frame by frame; wherein the image frames include partial images of the target image;

[0007] Segmentation step: segmenting the current image frame based on the i-th preset size among n different preset sizes to obtain an i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer;

[0008] Matching step: performing image matching on the i-th size image and an image frame preceding the current image frame to obtain an i-th matching degree between the i-th size image and the preceding image frame and an i-th offset of the i-th size image in the preceding image frame;

[0009] Fusion step: in response to the i-th matching degree being within a preset range, fusing the current image frame with the previous image frame based on the i-th offset.

[0010] In some embodiments, the method further comprises:

[0011] For every two adjacent image frames in the multiple image frames, the segmentation step, the matching step and the fusion step are performed until the multiple image frames are fused to obtain the target image.

[0012] In some embodiments, the method further comprises:

[0013] In response to the i-th matching degree being less than a lower limit of the preset range, determining whether i is equal to n;

[0014] In response to i not being equal to n, segmenting the current image frame based on an (i+1)th preset size to obtain an (i+1)th size image corresponding to the current image frame;

[0015] Performing image matching between the i+1th size image and the previous image frame, obtaining an i+1th matching degree between the i+1th size image and the previous image frame and an i+1th offset of the i+1th size image in the previous image frame;

[0016] In response to the i+1th matching degree being within a preset range, the current image frame is merged with the previous image frame based on the i+1th offset.

[0017] In some embodiments, the method further comprises:

[0018] In response to i being equal to n, the current image frame is merged with the previous image frame based on an offset corresponding to a maximum value among all matching degrees corresponding to the n different preset sizes.

[0019] In some embodiments, the method further comprises:

[0020] The speed and acceleration of each first image frame are calculated based on the displacement and frame rate between each two adjacent first image frames in the plurality of image frames that are located before the current image frame;

[0021] Predicting a predicted matching area in the current image frame that matches the previous image frame based on the speed and acceleration;

[0022] The priorities of the n preset sizes are determined based on the predicted matching area, so as to segment the current image frame based on the priority order of the preset sizes.

[0023] In some embodiments, fusing the current image frame with the previous image frame based on the i-th matching degree and the i-th offset includes:

[0024] In response to the i-th matching degree being less than a first threshold, fusing the current image frame with the previous image frame based on a first fusion method;

[0025] In response to the i-th matching degree being greater than or equal to a first threshold and less than a second threshold, the current image frame is fused with the previous image frame based on a second fusion method.

[0026] In some embodiments, the second fusion method includes a linear weighted fusion method, wherein when the i-th matching degree is in different intervals between the first threshold and the second threshold, the weight of the current image frame in the fusion with the previous image frame is different.

[0027] In some embodiments, the method further includes: in response to the i-th matching degree being greater than or equal to the second threshold, discarding the current image frame.

[0028] In some embodiments, the method further includes: performing fill light processing and / or spot removal processing when acquiring each of the image frames.

[0029] Based on the same inventive concept, the exemplary embodiment of the present disclosure further provides a device for acquiring a target image, comprising:

[0030] An acquisition module, used for acquiring a plurality of image frames about the target image frame by frame; wherein the image frames include partial images of the target image;

[0031] a segmentation module, configured to segment the current image frame based on an i-th preset size among n different preset sizes, to obtain an i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer;

[0032] a matching module, configured to perform image matching between the i-th size image and an image frame preceding the current image frame, and obtain an i-th matching degree between the i-th size image and the preceding image frame and an i-th offset of the i-th size image in the preceding image frame;

[0033] A fusion module is used for fusing the current image frame with the previous image frame based on the i-th offset in response to the i-th matching degree being within a preset range.

[0034] Based on the same inventive concept, an exemplary embodiment of the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for acquiring a target image as described in any one of the above items is implemented.

[0035] Based on the same inventive concept, an exemplary embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the target image acquisition method as described in any one of the above items.

[0036] Based on the same inventive concept, an exemplary embodiment of the present disclosure further provides a computer program product, including computer program instructions, which, when executed on a computer, enable the computer to execute any of the above methods for acquiring a target image.

[0037] From the above, it can be seen that the target image acquisition method and related equipment provided by the present disclosure divide the image frames collected frame by frame into different sizes, and match the image frame with the previous image frame based on the segmented image, and merge it with the previous image frame after the match is successful, so as to obtain the target image. Compared with the traditional method, the image matching based on the segmented image and the previous image frame has a smaller amount of calculation than that based on feature points, reduces the delay, and is not limited by the number of feature points contained in the image frame, thereby improving the effect of the obtained target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A schematic diagram of a target image acquisition architecture according to an exemplary embodiment of the present disclosure.

[0040] Figure 2 The present invention is a schematic diagram of an application scenario of a method for acquiring a target image according to an exemplary embodiment of the present invention.

[0041] Figure 3 The figure is a flowchart of a method for acquiring a target image according to an exemplary embodiment of the present disclosure.

[0042] Figure 4 FIG. 4 is a schematic diagram of image frame segmentation according to an exemplary embodiment of the present disclosure.

[0043] Figure 5 It is a schematic diagram of the offset of an exemplary embodiment of the present disclosure.

[0044] Figure 6 It is a schematic diagram of a predicted matching area of ​​an exemplary embodiment of the present disclosure.

[0045] Figure 7 It is a schematic structural diagram of a device for acquiring a target image according to an exemplary embodiment of the present disclosure.

[0046] Figure 8 It is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0048] According to an embodiment of the present disclosure, a method for acquiring a target image and related devices are proposed.

[0049] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0050] The principle and spirit of the present application are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0052] The solution disclosed in the present invention aims to provide a method for acquiring target content and related equipment to implement an improved solution for acquiring target images.

[0053] At present, when a user uses a device such as a dictionary pen equipped with an image acquisition device (such as a camera) to scan the content to be collected, the image acquisition device collects images of the content at the current position at a certain frame rate as the device moves, thereby obtaining multiple frames of images. The device uses the corresponding detection algorithm to detect the key feature points and invariant feature descriptors of each image (for example, the key feature points of the image are detected by algorithms such as Harris, and the invariant feature descriptors of the image are calculated by algorithms such as SIFT), and matches the key feature points and invariant feature descriptors of the two frames of images to obtain several matching point pairs, and removes false matches. Then, based on the matching point pairs, the images are affine transformed and then spliced, and the overlapping parts are fused until all images are fused to obtain a panoramic image of the content to be collected.

[0054] In the process of implementing the present disclosure, the inventors found that the above-mentioned prior arts all have obvious deficiencies. The traditional method of acquiring target images, based on key feature points for fusion and panoramic image stitching, results in a large amount of calculation, high latency, and reduced user experience. Due to the limitation of the device size, the field of view of the image acquisition device is small, and it may not be possible to obtain enough key feature points to support the algorithm, resulting in incorrect matching, making the final panoramic image effect poor.

[0055] Based on the characteristics of the above-mentioned target image acquisition process, and in view of the problems existing in the above-mentioned prior art, the present disclosure provides a method for acquiring a target image and related equipment, which divides the image frames collected frame by frame into different sizes, and matches the image after segmentation with the previous image frame, and merges it with the previous image frame after successful matching, so as to obtain the target image. Compared with the traditional method, the image matching using the segmented image and the previous image frame has a smaller amount of calculation than that based on feature points, reduces the delay, and is not limited by the number of feature points contained in the image frame, thereby improving the effect of the obtained target image.

[0056] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0057] Application Scenario Overview

[0058] refer to Figure 1 , which is a schematic diagram of the target image acquisition architecture provided by the embodiment of the present disclosure. The target image acquisition architecture 100 includes a server 110, a terminal 120, and a network 130 that provides a communication link. The server 110 and the terminal 120 can be connected via a wired or wireless network 130. Among them, the server 110 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0059] The terminal 120 may be implemented in hardware or software. For example, when the terminal 120 is implemented in hardware, it may be various electronic devices having a display screen and supporting page display, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, and desktop computers, etc. When the terminal 120 device is implemented in software, it may be installed in the electronic devices listed above; it may be implemented as multiple software or software modules (such as software or software modules used to provide distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.

[0060] It should be noted that the target content acquisition method provided in the embodiment of the present application can be executed by the terminal 120 or by the server 110. It should be understood that Figure 1 The number of terminals, networks and servers in the embodiment is only for illustration and is not intended to limit the number of terminals, networks and servers. Any number of terminals, networks and servers may be provided as required.

[0061] refer to Figure 2, which is a schematic diagram of an application scenario of the target image acquisition method provided by the embodiment of the present disclosure. Figure 2 As shown, the content 201 to be collected includes the string "Abcdefgh ijkl mnopq rstuvw xyz". It should be understood that the content 201 can also be an image, and there is no limitation on the content 201. The user uses the image acquisition device of the device to scan the content 201 to obtain m image frames 202a, 202b, 202c, ... 202m, where m is a positive integer. Among them, the image frame 202a includes the string "Abcde", the image frame 202b includes the string "defgh", the image frame 202c includes the string "h ijkl" ... the image frame 202m includes the string "xyz". The subsequent image frame can be spliced ​​to the previous image frame in sequence, that is, the same parts are merged and then spliced. Since the image frame 202a is the first frame image, it can be directly used for splicing. Image frame 202b can be spliced ​​to image frame 202a. Specifically, image frame 202b can be segmented in accordance with n preset sizes size1, size2, ... sizen, where n is a positive integer. After each segmentation, the segmented sub-image is image matched with the previous image frame 202a to calculate the matching degree and offset of each sub-image with the previous image frame 202a. When the matching degree meets the preset range, the segmentation can be stopped, and image frame 202b can be merged with image frame 202a based on the corresponding offset. For example, if the same part between image frame 202b and image frame 202a is the string "de", the string "de" part of image frame 202b can be merged with the string "de" part of image frame 202a to splice image frame 202b to image frame 202a. And so on, until all image frames 202a-202m are spliced, the final panoramic image is obtained as the target image 203 of content 201.

[0062] The method of the disclosed embodiment divides the image frames collected frame by frame into different sizes, matches the segmented image with the previous image frame, and fuses it with the previous image frame after the match is successful, thereby obtaining a target image. Compared with the traditional method, the image matching using the segmented image and the previous image frame has a smaller amount of calculation than that based on feature points, reduces delay, and is not limited by the number of feature points contained in the image frame, thereby improving the effect of the obtained target image.

[0063] Combine the following Figure 2 The target content acquisition method according to the exemplary embodiment of the present disclosure is described by using the application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0064] Exemplary Methods

[0065] First, the embodiment of the present disclosure provides a method for acquiring a target image, which can be executed by a server, such as Figure 1 The server 110 in FIG. 110 may also be executed by the client, such as Figure 1 The client 120 in the embodiment may also be executed by the server and the client together, for example, the client 120 may obtain multiple image frames about the target image, send the multiple image frames to the server 110 via the network 130, and the server 110 may perform the segmentation step, the matching step and the fusion step of the multiple image frames to obtain the final target image; and then the target image may be sent to the client 120. Figure 3 The target image acquisition method 300 of the embodiment of the present disclosure may further include the following steps.

[0066] In step S301 , a plurality of image frames about the target image are acquired frame by frame; wherein the image frames include partial images of the target image.

[0067] The image acquisition device may acquire content at a fixed frequency during scanning to obtain a plurality of corresponding image frames. Each image frame may include a partial image of the target image corresponding to the content to be acquired. For example, Figure 1 In the example, the image acquisition device scans the character string “Abcdefgh ijkl mnopq rstuvw xyz” to obtain a plurality of image frames 202a-202m, wherein the image frames 202a-202m each include a partial image of the target image 203.

[0068] In some embodiments, the method 300 further includes: performing fill light processing and / or spot removal processing when acquiring the image frame.

[0069] Among them, if the target image obtained after the subsequent fusion and splicing of the image frame is too dark, it will affect the subsequent target object detection and target object recognition, such as text detection and text recognition. Therefore, the image acquisition device will set a light source for fill light while acquiring the image frame. If the carrier of the content to be acquired is a relatively smooth surface, then there may be a relatively obvious light spot on each image frame during scanning, so each image frame can be processed for light spots to ensure the clarity of the image frame, thereby improving the accuracy of subsequent fusion and splicing.

[0070] In step S302, the segmentation step is: segmenting the current image frame based on the i-th preset size among n different preset sizes to obtain the i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer.

[0071] The scanning speed of the image acquisition device is different, and the position of the overlapping area in two adjacent image frames is different. For example, the faster the scanning speed, the smaller the similarity of adjacent image frames; conversely, the greater the similarity of adjacent image frames. The n preset sizes for segmentation can be set according to the frame rate and field of view of the data acquisition device, such as size1, size2, ..., sizen, where n is a positive integer. Figure 4 As shown, Figure 4 A schematic diagram of image frame segmentation according to an embodiment of the present disclosure is shown. Figure 4 In Figure 2 The image frame 202b shown is segmented with a first preset size size1 to obtain a corresponding first size image 401. The first size image 401 can be matched with the previous image frame 202a to obtain a corresponding first matching degree. If the preset requirements are met in the first matching degree, the image frame 202b is stopped from being segmented with a second preset size size2. If the preset requirements are not met in the first matching degree, the image frame 202b is continued to be segmented with a second preset size size2 to obtain a second size image (not shown). Similarly, if the i-th matching degree corresponding to the i-th preset size meets the preset requirements, the image frame 202b is stopped from being segmented with other preset sizes; if the i-th matching degree does not meet the preset requirements, the image frame 202b is continued to be segmented with other preset sizes until the matching degree of a certain size segmented image meets the preset requirements with the previous image frame or all preset sizes have been segmented.

[0072] Further, in some embodiments, when the current image frame is segmented based on the i-th preset size, it may be based on a preset rule. In some embodiments, the preset rule may be a preset area for segmenting the current image frame. For example, the first preset area (e.g., the first area of ​​the i-th size located behind the left edge) may be segmented based on the left edge of the current image frame. Figure 4 The second preset area on the left side of the center line and / or the third preset area on the right side of the center line (such as 401 in the example) may be segmented based on the center line of the current image frame. Figure 4 402, 403 in the figure) as the i-th size image. Alternatively, the center point of the current image frame may be used as a reference to segment a fourth preset area (not shown) centered on the center point. It should be understood that the above preset area is only an example and is not intended to limit the preset area. The position of the preset area can be set as needed, and is not limited here.

[0073] In some embodiments, the preset sizes may have priorities, and the image frame may be segmented according to the priority order of the n preset sizes.

[0074] In step S303, a matching step is performed: the i-th size image is matched with an image frame previous to the current image frame to obtain an i-th matching degree between the i-th size image and the previous image frame and an i-th offset of the i-th size image in the previous image frame.

[0075] For each segmentation, the size image obtained by segmentation can be similarly matched with the previous image frame. If the matching degree is greater, the similarity between the size image and the previous image frame is greater; if the matching degree is smaller, the similarity between the size image and the previous image frame is smaller. On this basis, the offset between the size image and the previous image frame can also be calculated. For example, Figure 4 In the example, after the image frame 202b is segmented into the first preset size size1, a first matching degree S between the first size image 401 and the previous image frame 202a is calculated. 401 Furthermore, the first offset Δw of the first size image 401 relative to the previous image frame 202a can also be obtained: 401 .like Figure 5 As shown, Figure 5 A schematic diagram showing an offset according to an embodiment of the present disclosure is shown. Figure 5 In the example, the upper left corner of the previous image frame 202a is taken as the origin (0, 0) of the coordinate system. When the first size image 401a is matched with the previous image frame 202a, the two are aligned based on the overlapping area. After the alignment, the upper left corner of the first size image 401a is recorded as point A in the coordinate system. The distance from point A to the origin (0, 0) on the horizontal coordinate is the first offset △w 401 It should be understood that the calculation of the i-th offset between the i-th size image and the previous image frame is similar to that of the first size image 401, and will not be repeated here.

[0076] Further, in some embodiments, the speed and acceleration of each first image frame may be obtained by calculating based on the displacement and frame rate between each two adjacent first image frames in the plurality of first image frames that are located before the current image frame;

[0077] Predicting a predicted matching area in the current image frame that matches the previous image frame based on the speed and acceleration;

[0078] The priorities of the n preset sizes are determined based on the predicted matching area, so as to segment the current image frame based on the priority order of the preset sizes.

[0079] Among them, the displacement corresponding to the i-th size image with a matching degree within the preset range can be used as the displacement between the current image frame and the previous image frame. The speed and acceleration during scanning can be calculated through the offset between multiple image frames, so as to predict the motion trajectory during scanning and infer the possible position of the next image frame. In this way, the priority of the preset size during segmentation can be dynamically adjusted, so as to speed up the segmentation step, further reduce the amount of calculation and reduce the delay. For example, Figure 2 The displacement between image frame 202b and image frame 202a is △wba, the displacement between image frame 202c and image frame 202b is △wcb, and the frame rate of the image acquisition device is f. Then, it can be calculated that the speed vb of image frame 202b is △wba*f, the speed vc of image frame 202c is △wcb*f, and the acceleration a from image frame 202b to image frame 202c is (vc-vb)*f. Based on this, the position of the matching area between image frame 202d and image frame 202c can be predicted. For example, it can be predicted that the speed of image frame 202d is vd=vc+a / f, and the displacement between image frame 202d and image frame 202c is △wdc=vd / f. Since the width of each image frame is consistent and fixed, assuming the width is w, it can be determined that the predicted matching area between image frame 202d and image frame 202c is the area w-△wdc in the second half of image frame 202c, such as Figure 6 As shown, Figure 6 A schematic diagram of a predicted matching area according to an embodiment of the present disclosure is shown. Figure 6 In the example, the predicted matching area of ​​image frame 202d and image frame 202c includes the string "kl", and the priority of the preset size can be adjusted according to the size w-△wdc of the area. Because the predicted matching area is very likely to be the overlapping area of ​​image frame 202d and image frame 202c, the segmented image (such as the i-th size image) with a size closer to the predicted matching area has a higher matching degree and is more likely to meet the preset range of matching degree. This can speed up the segmentation step and further reduce the delay of the entire target image acquisition. The priority of the preset size that is closer to the size of the predicted matching area can be increased. For example, the difference between the size of the predicted matching area and each preset size can be calculated, and the priority order of the preset sizes can be set from small to large according to the difference. The smaller the difference, the closer the size of the predicted matching area is to the preset size, and the higher the priority of the preset size; the larger the difference, the smaller the size of the predicted matching area is to the preset size, and the lower the priority of the preset size.

[0080] At step S304, a fusion step: in response to the i-th matching degree being within a preset range, the current image frame is fused with the previous image frame based on the i-th offset.

[0081] Among them, the i-th predicted size is used as a cropping template to perform template matching with the current image frame. The template matching can be a traditional template matching method or a pyramid-based template matching method. In this way, compared with the prior art that uses key feature points for matching, the amount of template matching calculation is greatly reduced, thereby speeding up the acquisition of the target image and reducing delays. Specifically, the i-th predicted size is used as a cropping template to crop the current image frame to obtain a corresponding i-th size image, and the i-th size image is template matched with the previous image frame to obtain a corresponding i-th matching degree. When the i-th matching degree is within a preset range, fusion can be performed.

[0082] In some embodiments, method 300 further includes:

[0083] In response to the i-th matching degree being less than a lower limit of the preset range, determining whether i is equal to n;

[0084] In response to i not being equal to n, segmenting the current image frame based on an (i+1)th preset size to obtain an (i+1)th size image corresponding to the current image frame;

[0085] Performing image matching between the i+1th size image and the previous image frame, obtaining an i+1th matching degree between the i+1th size image and the previous image frame and an i+1th offset of the i+1th size image in the previous image frame;

[0086] In response to the i+1th matching degree being within a preset range, the current image frame is merged with the previous image frame based on the i+1th offset.

[0087] In some embodiments, method 300 further includes:

[0088] In response to i being equal to n, the current image frame is merged with the previous image frame based on an offset corresponding to a maximum value among all matching degrees corresponding to the n different preset sizes.

[0089] Among them, when the i-th matching degree corresponding to the i-th size image corresponding to the current image frame is within the preset range, it is determined whether i is n at this time, that is, it is determined whether the current image frame is segmented based on all preset sizes. If i is not n, it means that the current image frame is not segmented based on all preset sizes, and the segmentation step can be continued based on the next predicted size (for example, the i+1th predicted size). If i is n, it means that the current image frame has been segmented based on all preset sizes. At this time, the maximum matching degree among all matching degrees corresponding to the 1st-nth size images obtained by segmentation of all preset sizes can be taken, and the offset corresponding to the maximum matching degree is used as the basis for the fusion of the current image frame with the previous image frame.

[0090] In some embodiments, method 300 further includes:

[0091] The method further comprises: fusing the current image frame with the previous image frame based on the i-th matching degree and the i-th offset, comprising:

[0092] In response to the i-th matching degree being less than a first threshold, fusing the current image frame with the previous image frame based on a first fusion method;

[0093] In response to the i-th matching degree being greater than or equal to a first threshold and less than a second threshold, the current image frame is fused with the previous image frame based on a second fusion method.

[0094] Among them, for different matching degrees, different fusion methods can be used, such as linear weighted fusion or Poisson fusion, to improve the effect and efficiency of the fusion. Specifically, the first fusion method can be used when the matching degree is less than the first threshold; the second fusion method can be used when the matching degree is greater than or equal to the first threshold and less than the second threshold. Further, the first fusion method may be different from the second fusion method. Furthermore, in some embodiments, the first fusion method includes a Poisson fusion method. For example, the first threshold may be 70 points, then when the matching degree is less than 70, the Poisson fusion method is used.

[0095] In some embodiments, method 300 further includes:

[0096] The second fusion method comprises a linear weighted fusion method, wherein when the i-th matching degree is in different intervals between the second threshold and the third threshold, the weight of the current image frame in the fusion with the previous image frame is different.

[0097] Among them, for the linear weighted fusion method, different weights will be used for different matching degrees. For example, the second threshold can be 95 points, then the first threshold and the second threshold are between (70,95), and there can be multiple different intervals between the first threshold and the second threshold. For example, the interval is divided by a preset interval (for example, 5 points). The matching degree is above 70 points, that is, greater than the first threshold, and the linear weighted fusion method can be used. Then set the weighted value in segments, for example, the matching degree is 70-75 corresponding to the first group of weights (for example, the weight of the current image frame is w1, and the weight of the previous image frame is w2), and the matching degree is 76-80 corresponding to the second group of weights (for example, the weight of the current image frame is w3, and the weight of the previous image frame is w4), and so on. In this way, different fusion methods can be selected based on different matching degrees, and a clearer panoramic target image can be obtained.

[0098] In some embodiments, the method 300 further includes: in response to the i-th matching degree being greater than or equal to the second threshold, discarding the current image frame.

[0099] Different matching degrees have different meanings. If the matching degree is extremely high, for example, higher than the second threshold of 95 points, it may be that the moving distance during scanning is small or not moving, and the current frame can be selectively discarded. In this way, unnecessary calculations can be avoided, the acquisition efficiency of the target image can be further improved, and the delay can be reduced.

[0100] In some embodiments, method 300 further includes:

[0101] For every two adjacent image frames in the multiple image frames, the segmentation step, the matching step and the fusion step are performed until the multiple image frames are fused to obtain the target image.

[0102] Specifically, as image frames are continuously captured, the segmentation step, matching step, and fusion step of the embodiment of the present disclosure can be executed in real time for every two adjacent image frames until the scanning is completed, and the segmentation step, matching step, and fusion step of the embodiment of the present disclosure are executed for all adjacent image frames, thereby obtaining a panoramic target image of the content to be captured.

[0103] In some embodiments, method 300 further includes: performing target recognition based on the target image. Further, target recognition includes text recognition and / or image recognition. For example, when a user obtains Figure 2 After the target image 203 shown in the image is obtained, the character string in the target image 203 can be identified. Furthermore, a search can be performed in the corresponding database to return the meaning corresponding to the character string (e.g., Chinese and / or foreign interpretations, etc.). In this way, according to the embodiment of the present disclosure, when the image acquisition device is a dictionary pen, it can quickly and effectively help users accurately identify the content to be searched, and return the query results, thereby achieving low-latency, efficient and accurate query, and improving user experience.

[0104] Exemplary Devices

[0105] refer to Figure 7 Based on the same inventive concept as any of the above-mentioned methods for acquiring a target image, the present disclosure also provides a device for acquiring a target image. The device for acquiring a target image comprises:

[0106] An acquisition module, used for acquiring a plurality of image frames about the target image frame by frame; wherein the image frames include partial images of the target image;

[0107] a segmentation module, configured to segment the current image frame based on an i-th preset size among n different preset sizes, to obtain an i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer;

[0108] a matching module, configured to perform image matching between the i-th size image and an image frame preceding the current image frame, and obtain an i-th matching degree between the i-th size image and the preceding image frame and an i-th offset of the i-th size image in the preceding image frame;

[0109] A fusion module is used for fusing the current image frame with the previous image frame based on the i-th offset in response to the i-th matching degree being within a preset range.

[0110] In some embodiments, for every two adjacent image frames among the multiple image frames, the segmentation module is also used to perform the segmentation step, the matching module is also used to perform the matching step, and the fusion module is also used to perform the fusion step, until the fusion of the multiple image frames is completed to obtain the target image.

[0111] In some embodiments, the apparatus further comprises:

[0112] A judging module, configured to judge whether i is equal to n in response to the i-th matching degree being less than a lower limit of the preset range;

[0113] The segmentation module is further used for: in response to i not being equal to n, segmenting the current image frame based on the i+1th preset size to obtain an i+1th size image corresponding to the current image frame;

[0114] The matching module is further used to: perform image matching based on the i+1th size image and the previous image frame to obtain the i+1th matching degree between the i+1th size image and the previous image frame and the i+1th offset of the i+1th size image in the previous image frame;

[0115] The fusion module is further configured to: in response to the i+1th matching degree being within a preset range, fuse the current image frame with the previous image frame based on the i+1th offset.

[0116] In some embodiments, the fusion module is further used to: in response to i being equal to n, fuse the current image frame with the previous image frame based on an offset corresponding to a maximum value among all matching degrees corresponding to the n different preset sizes.

[0117] In some embodiments, the apparatus further comprises:

[0118] A calculation module, configured to calculate the speed and acceleration of each first image frame based on the displacement and frame rate between each two adjacent first image frames in the plurality of first image frames that are located before the current image frame;

[0119] A prediction module, configured to predict a predicted matching area in the current image frame that matches the previous image frame based on the speed and acceleration;

[0120] The segmentation module is further used to determine the priorities of the n preset sizes based on the predicted matching area, so as to segment the current image frame based on the priority order of the preset sizes.

[0121] In some embodiments, the fusion module is further configured to:

[0122] In response to the i-th matching degree being less than a first threshold, fusing the current image frame with the previous image frame based on a first fusion method;

[0123] In response to the i-th matching degree being greater than or equal to a second threshold and less than a third threshold, the current image frame is fused with the previous image frame based on a second fusion method.

[0124] In some embodiments, the second fusion method comprises a linear weighted fusion method, wherein when the i-th matching degree is in different intervals between the second threshold and the third threshold, the weight of the current image frame in the fusion with the previous image frame is different.

[0125] In some embodiments, the fusion module is further used to: in response to the i-th matching degree being greater than or equal to the third threshold, discard the current image frame.

[0126] In some embodiments, the apparatus further comprises:

[0127] The pre-processing module is used to perform fill light processing and / or spot removal processing when acquiring each of the image frames.

[0128] The device of the above embodiment is used to implement the corresponding target image acquisition method in any embodiment of the above exemplary target image acquisition method part, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0129] Based on the same inventive concept as any of the above-mentioned methods for acquiring a target image, an embodiment of the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for acquiring the target content described in any of the above embodiments is implemented.

[0130] Figure 8The electronic device structure schematic diagram of an exemplary embodiment of the present disclosure is shown, and the device may include: a processor 810, a memory 820, an input / output interface 830, a communication interface 840, and a bus 850. The processor 810, the memory 820, the input / output interface 830, and the communication interface 840 are connected to each other through the bus 850 in the device.

[0131] The processor 810 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0132] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 820 and called and executed by the processor 810.

[0133] The input / output interface 830 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0134] The communication interface 840 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0135] The bus 850 comprises a pathway that transmits information between the various components of the device (eg, the processor 810, the memory 1020, the input / output interface 830, and the communication interface 840).

[0136] It should be noted that, although the above device only shows the processor 810, the memory 820, the input / output interface 830, the communication interface 840 and the bus 850, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0137] The electronic device of the above embodiment is used to implement the corresponding target content acquisition method in any embodiment of the above exemplary method part, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0138] Exemplary Program Products

[0139] Based on the same inventive concept as any of the above-mentioned target content acquisition method embodiments, the present disclosure embodiment also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the corresponding target content acquisition method in any of the embodiments in the aforementioned exemplary method part.

[0140] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0141] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the target image acquisition method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0142] It is known to those skilled in the art that the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present invention may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0143] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0144] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0146] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0148] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0149] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0150] In addition, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0151] The use of the verbs "comprise", "include" and their conjugations mentioned in the application documents does not exclude the presence of elements or steps other than those recorded in the application documents. The article "a" or "an" before an element does not exclude the presence of a plurality of such elements.

[0152] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and this division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A method for acquiring a target image, characterized in that: include: Acquire multiple image frames about the target image frame by frame; wherein the image frames include partial images of the target image; Segmentation step: segmenting the current image frame based on the i-th preset size among n different preset sizes to obtain an i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer; Matching step: performing image matching on the i-th size image and an image frame preceding the current image frame to obtain an i-th matching degree between the i-th size image and the preceding image frame and an i-th offset of the i-th size image in the preceding image frame; A fusion step: in response to the i-th matching degree being within a preset range, fusing the current image frame with the previous image frame based on the i-th offset; The method further includes: calculating the speed and acceleration of each first image frame based on the displacement and frame rate between each two adjacent first image frames in the plurality of first image frames before the current image frame; predicting a predicted matching area in the current image frame that matches the previous image frame based on the speed and acceleration; determining the priorities of the n preset sizes based on the predicted matching area, and segmenting the current image frame based on the priority order of the preset sizes; For every two adjacent image frames in the multiple image frames, the segmentation step, the matching step and the fusion step are performed until the multiple image frames are fused to obtain the target image.

2. The method according to claim 1, characterized in that Also includes: In response to the i-th matching degree being less than a lower limit of the preset range, determining whether i is equal to n; In response to i not being equal to n, segmenting the current image frame based on an (i+1)th preset size to obtain an (i+1)th size image corresponding to the current image frame; Performing image matching between the i+1th size image and the previous image frame, obtaining an i+1th matching degree between the i+1th size image and the previous image frame and an i+1th offset of the i+1th size image in the previous image frame; In response to the i+1th matching degree being within a preset range, the current image frame is merged with the previous image frame based on the i+1th offset.

3. The method according to claim 2, characterized in that Also includes: In response to i being equal to n, the current image frame is merged with the previous image frame based on an offset corresponding to a maximum value among all matching degrees corresponding to the n different preset sizes.

4. The method according to claim 1, characterized in that The method further comprises: fusing the current image frame with the previous image frame based on the i-th matching degree and the i-th offset, comprising: In response to the i-th matching degree being less than a first threshold, the current image frame is fused with the previous image frame based on a first fusion method; in response to the i-th matching degree being greater than or equal to the first threshold and less than a second threshold, the current image frame is fused with the previous image frame based on a second fusion method.

5. The method according to claim 4, characterized in that The second fusion method comprises a linear weighted fusion method, wherein when the i-th matching degree is in different intervals between the first threshold and the second threshold, the weight of the current image frame in the fusion with the previous image frame is different.

6. The method according to claim 4, characterized in that include: In response to the i-th matching degree being greater than or equal to the second threshold, discarding the current image frame.

7. The method according to claim 1, characterized in that Also includes: When acquiring each of the image frames, fill light processing and / or spot removal processing is performed.

8. A device for acquiring a target image, characterized in that: include: An acquisition module, used for acquiring a plurality of image frames about the target image frame by frame; wherein the image frames include partial images of the target image; a segmentation module, configured to segment the current image frame based on an i-th preset size among n different preset sizes, to obtain an i-th size image corresponding to the current image frame; wherein i is an integer greater than or equal to 1 and less than or equal to n, and n is a positive integer; a matching module, configured to perform image matching between the i-th size image and an image frame preceding the current image frame, and obtain an i-th matching degree between the i-th size image and the preceding image frame and an i-th offset of the i-th size image in the preceding image frame; a fusion module, configured to fuse the current image frame with the previous image frame based on the i-th offset in response to the i-th matching degree being within a preset range; The device is further used for: calculating the speed and acceleration of each first image frame based on the displacement and frame rate between each two adjacent first image frames in the multiple image frames that are located before the current image frame; predicting the predicted matching area in the current image frame that matches the previous image frame based on the speed and acceleration; determining the priorities of n preset sizes based on the predicted matching area, and segmenting the current image frame based on the priority order of the preset sizes; and executing the steps of the segmentation module, the matching module and the fusion module for each two adjacent image frames in the multiple image frames until the fusion of the multiple image frames is completed to obtain the target image.

Images