Page turning detection method and device, electronic equipment and storage medium

Abstract

The application provides a page turning detection method and device, electronic equipment and storage medium, the method comprises: identifying each frame of image in the image sequence to be detected as a stable image or a motion image; for each stable image, determining a first change area of the stable image compared with a reference image, and a second change area of the previous stable image of the stable image compared with the reference image; the reference image is a motion image arranged between the stable image and the previous stable image in the image sequence; based on the first change area and the second change area, determine the page turning detection result of the stable image. The method, device, electronic equipment and storage medium provided by the application accurately compare the picture change areas before and after the action interference, thereby effectively filtering the disturbance of various instantaneous interference factors on the page turning detection, realizing the page identity verification, avoiding the problem of same page reset misjudgment, and greatly improving the anti-interference ability, robustness and accuracy of the page turning detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a page-turning detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, page-turning detection is usually implemented using a hash algorithm. Specifically, it can be determined whether a page turn has occurred by calculating and comparing the global hash difference between the images of pages in the preceding and following frames.

[0003] However, in real-world applications, factors such as waving hands, book movement, environmental shaking, or even insects flying by can all cause changes in the global hash of the page image, leading to a significant difference in the global hash between consecutive frames. Consequently, these pages are frequently misidentified as page turns during page turn detection. The weak anti-interference capability of page turn detection schemes directly impacts the accuracy of human-computer interaction and user experience based on page turn detection. Summary of the Invention

[0004] This invention provides a page-turning detection method, apparatus, electronic device, and storage medium to address the shortcomings of related technologies, such as poor anti-interference capability and susceptibility to misjudgment in page-turning detection.

[0005] This invention provides a page-turning detection method, comprising: Identify whether each frame in the image sequence to be detected is a stable image or a moving image; For each frame of a stable image, a first region of change in the stable image compared to a reference image is determined, and a second region of change in the previous stable image of the stable image compared to the reference image is determined; the reference image is a motion image arranged in the image sequence between the stable image and the previous stable image. Based on the first change region and the second change region, the page-turning detection result of the stable image is determined.

[0006] According to a page-turning detection method provided by the present invention, determining the first change region of the stable image compared to the reference image, and the second change region of the previous stable image of the stable image compared to the reference image, includes: If the difference in image features between the stable image and the previous stable image is greater than or equal to a first threshold, a first change region of the stable image compared to the reference image and a second change region of the previous stable image compared to the reference image are determined.

[0007] The page-turning detection method provided by the present invention further includes: If the difference in image features between the stable image and the previous stable image is less than the first threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0008] According to a page-turning detection method provided by the present invention, determining the page-turning detection result of the stable image based on the first change region and the second change region includes: Determine the regional feature differences between the first and second change regions; If the difference in regional features is greater than or equal to the second threshold, the page-turning detection result of the stable image is determined to be page-turned; If the difference in regional features is less than the second threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0009] According to the page-turning detection method provided by the present invention, the step of identifying each frame of the image sequence to be detected as a stable image or a moving image includes: Detect the image change state between every two adjacent frames in the image sequence; Based on the image change state between every two adjacent frames, each frame is determined to be either a stable image or a moving image.

[0010] According to a page-turning detection method provided by the present invention, determining whether each frame image is a stable image or a moving image based on the image change state between every two adjacent frames includes: For each frame of an image, a preset number of consecutive adjacent images are determined with the last frame of that image. If there is no change in the image state between consecutive adjacent images in the preset number of groups, the image is determined to be a stable image.

[0011] The present invention also provides a page-turning detection device, comprising: The state recognition unit is used to identify whether each frame in the image sequence to be detected is a stable image or a moving image; The feature extraction unit is used to determine, for each frame of stable image, a first change region of the stable image compared to a reference image, and a second change region of the previous stable image of the stable image compared to the reference image; the reference image is a motion image arranged in the image sequence between the stable image and the previous stable image; The page-turning detection unit is used to determine the page-turning detection result of the stable image based on the first change region and the second change region.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the page turning detection method as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the page-turning detection method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the page-turning detection method as described above.

[0015] The page-turning detection method, apparatus, electronic device, and storage medium provided by this invention introduce a motion image in an intermediate transition state as a reference image, thereby determining the first change region of the stable image compared to the reference image, and the second change region of the previous stable image compared to the reference image, achieving precise positioning of the image change region before and after motion interference. Therefore, page-turning detection based on the first and second change regions reduces the traditional blind comparison of the entire frame to a precise comparison of the image change region before and after motion interference, effectively filtering out various instantaneous interference factors that disturb page-turning detection, achieving page identity verification, avoiding the problem of misjudgment due to same-page resetting, and significantly improving the anti-interference capability, robustness, and accuracy of page-turning detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the page-turning detection method provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the page-turning detection device provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0022] With the development of artificial intelligence technology, more and more people are using smart reading devices to assist in reading physical books. Here, smart reading devices can include picture book reading robots, learning machines, etc. Taking picture book reading robots as an example, as a companion-style picture book reading aid, the core interactive experience of a picture book reading robot relies on the precise synchronization between the robot and the reading progress, and accurate page-turning detection is the key technology to achieve this synchronization.

[0023] Currently, page-turn detection is typically implemented using hash algorithms. Specifically, a global hash value is calculated from the page images captured by the device, and the difference in global hash values ​​between consecutive frames is compared to determine if a page turn has occurred. While this approach is simple to implement and consumes little computational power, it relies solely on single-frame global feature comparison and lacks dynamic process analysis, resulting in the following drawbacks in practical applications: First, the aforementioned page-turning detection scheme lacks the ability to distinguish states. This scheme crudely determines whether a page image has changed by comparing global features in a single frame, directly equating this to whether a page has been turned. It cannot actually differentiate between states such as "page turning," "interference occurring," and "stable state" in actual book reading.

[0024] Secondly, the lack of dynamic process analysis and over-reliance on global hash values ​​result in extremely poor anti-interference capabilities for the aforementioned page-turning detection scheme. In real reading scenarios, non-page-turning actions such as waving hands, book movement, environmental shaking, or even insects flying by are very common. These instantaneous disturbances can trigger drastic changes in the global hash value of the page image, leading to frequent misjudgments as page-turning actions and severely damaging the interactive experience of the picture book reading robot.

[0025] Furthermore, the large difference in global hashes between consecutive page images could be due to non-page-turning book movement. The aforementioned page-turning detection scheme cannot verify whether consecutive page images correspond to the same page, and may directly classify it as a page turn.

[0026] In response, this invention provides a page-turning detection method. Figure 1 This is a flowchart illustrating the page-turning detection method provided by the present invention. Figure 1 As shown, this method can be applied to electronic devices with image acquisition and processing capabilities, such as picture book reading robots, learning machines, smart desk lamps, and tablet computers. The method includes: Step 110: Identify whether each frame in the image sequence to be detected is a stable image or a moving image.

[0027] Specifically, during operation, electronic devices continuously capture images of the reading area of ​​a physical book using their built-in camera or a pre-connected camera, thereby obtaining an image sequence.

[0028] Here, the physical book reading area is a pre-defined area for reading physical books. Typically, a physical book can be placed in this area and read, allowing the electronic device to detect page-turning actions in real time during the reading process. The physical book can be a picture book, magazine, pictorial, textbook, workbook, or other different types of reading material; this embodiment of the invention does not specifically limit the type.

[0029] It is understood that an image sequence is a sequence of multiple frames of images continuously acquired over time. The image sequence records the real-time view of the reading area of ​​a physical book over a period of time. Each frame in the image sequence is an image acquired specifically for the reading area of ​​the physical book. In the image sequence, any frame may or may not contain the physical book with the page-turning action to be detected; this embodiment of the invention does not specifically limit this.

[0030] After obtaining the image sequence to be detected, it may be necessary to classify the state of each frame in the image sequence, that is, to identify whether each frame is a stable image or a moving image.

[0031] Among them, a stable image refers to an image in which the content remains relatively still and does not undergo significant visual changes over time. It usually represents a physical book page in a static state being read quietly. A moving image, on the other hand, refers to an image in which the content is undergoing significant visual changes. It usually represents a physical book page being turned, or the presence of interfering factors such as the user's arm waving, the book moving, or a mosquito flying by.

[0032] Here, identifying each frame in the image sequence as a stable image or a moving image can be achieved by comparing whether each frame changes with the previous frame in the image sequence; or, it can be achieved by comparing whether each frame is consistent with the previous N frames in the image sequence, where N can be an integer greater than or equal to 2. The specific implementation method for identifying images as stable images or moving images in this embodiment of the invention is not limited.

[0033] Step 120: For each frame of stable image, determine the first change region of the stable image relative to the reference image, and the second change region of the previous stable image of the stable image relative to the reference image; the reference image is the motion image arranged in the image sequence between the stable image and the previous stable image.

[0034] Specifically, after distinguishing between stable and moving images for each frame in the image sequence, all stable images can be identified from the image sequence. Based on this, for each identified stable image, it can be further determined whether the stable image is the result of a page-turning action; that is, whether there is a page-turning action between the stable image and the previous stable image arranged before it in the image sequence.

[0035] For each stable image in an image sequence, the previous stable image refers to the image frame that was in a stable state most recently, preceding the current stable image. For ease of explanation, the stable image currently undergoing page-turning detection will be referred to as the current stable image. It can be understood that each stable image in the image sequence can be used as the current stable image for subsequent operations.

[0036] In a typical physical book reading scenario, there may be a transition period between two stable frames where an action occurs. In other words, in an image sequence, between the previous stable image and the current stable image, there may be moving images where the scene changes, and the number of moving images between the previous and current stable images can be one frame or more. In this case, any moving image between two stable frames can be used as a reference image. It can be understood that the reference image records the state of the scene when the action occurs.

[0037] Based on this, in order to determine whether the pages displayed on the screen have actually changed after the action, after determining the reference image, the first change area of ​​the current stable image compared to the reference image and the second change area of ​​the previous stable image compared to the reference image can be determined respectively.

[0038] The first change region refers to the set of spatial regions with visual differences extracted after comparing the current stable image with the reference image. It can also be understood as the set of regions in the current stable image that have changed compared to the reference image. The second change region refers to the set of spatial regions with visual differences extracted after comparing the previous stable image with the reference image. It can also be understood as the set of regions in the previous stable image that have changed compared to the reference image.

[0039] For example, the previous stable image is a complete view of the first page of a book. As time passes, the user's hand reaches into the image and obscures the bottom right corner of the book. The captured image at this point is marked as a moving image. Subsequently, the user removes their hand, and the image returns to stillness. The captured image at this point is the current stable image. In this case, the moving image where the hand obscures the bottom right corner of the book is used as the reference image. Comparing the current stable image with the reference image yields the first area of ​​change, which is the area in the bottom right corner that was obscured by the hand. Similarly, comparing the previous stable image with the reference image yields the second area of ​​change, which is also the area in the bottom right corner that was obscured by the hand.

[0040] In step 120, by introducing a reference image as a reference point, the effective spatial region where motion interference actually occurred can be accurately located between two adjacent stable frames, namely the first change region and the second change region. It can be understood that the first change region is the effective spatial region before the motion interference, and the second change region is the effective spatial region after the motion interference ended.

[0041] Furthermore, it should be noted that steps 120 and subsequent steps 130 are for cases where there are moving images between two adjacent stable frames. If there are no moving images between two adjacent stable frames, it can be directly determined that there is no page-turning action within the time period corresponding to the two stable frames, and there is no need to perform the page-turning detection steps 120 and 130.

[0042] Step 130: Based on the first change region and the second change region, determine the page-turning detection result of the stable image.

[0043] Specifically, for each stable image in the image sequence, by comparing the first and second change regions, it can be determined whether a page-turning action exists between the previous and current stable images, thus obtaining the page-turning detection result for the current stable image. Here, the page-turning detection result can be either "page-turned" or "page-not-turned." It is understood that "page-turned" and "page-not-turned" are relative to the previous stable image.

[0044] Specifically, the first and second change regions, as the difference regions extracted relative to the reference image, are compared by comparing the image content contained in each region. Essentially, this compares the changes in image content before and after the motion interference, or the changes in image content caused by the motion interference. If the first and second change regions are nearly identical, the motion interference can be considered instantaneous and does not affect the turning of the pages of the physical book. For example, it might just be an arm waving, environmental shaking, a mosquito flying by, or the book moving and then returning to its original position. In this case, the previous stable image and the current stable image reflect the same page content, meaning the previous stable image and the current stable image have page identity. Conversely, if there is a significant difference between the first and second change regions, it can be understood that the motion interference has caused a substantial replacement or change in the image content. It can be considered that the original page content has been covered by a new page, i.e., a page-turning action has occurred.

[0045] Therefore, based on the first and second change regions, accurate page-turning detection can be achieved for the current stable image.

[0046] In the method provided in this embodiment of the invention, by introducing a motion image in an intermediate transition state as a reference image, a first change region of the stable image compared to the reference image and a second change region of the previous stable image compared to the reference image are determined, achieving precise positioning of the image change region before and after motion interference. Therefore, page-turning detection is performed based on the first and second change regions, reducing the traditional blind comparison of the entire frame to a precise comparison of the image change region before and after motion interference. This effectively filters out various instantaneous interference factors from affecting page-turning detection, achieving page identity verification, avoiding the problem of misjudgment due to same-page reset, and significantly improving the anti-interference capability, robustness, and accuracy of page-turning detection.

[0047] Based on the above embodiments, determining the first change region of the stable image relative to the reference image, and the second change region of the previous stable image of the stable image relative to the reference image, includes: If the difference in image features between the stable image and the previous stable image is greater than or equal to a first threshold, a first change region of the stable image compared to the reference image and a second change region of the previous stable image compared to the reference image are determined.

[0048] Specifically, for each stable image in an image sequence, when determining whether the current stable image is the result of a page-turning action, or when determining whether there is a page-turning action between the previous stable image and the current stable image, the difference in image features between the current stable image and the previous stable image can be judged first.

[0049] Here, image feature difference refers to the change or difference in image features between two adjacent stable images in the global dimension. For the current stable image and the previous stable image, the image features can be global hash values, global color histogram differences, or feature vectors extracted by a pre-trained image feature model, etc. The image feature difference can be the difference between the image features of the two images, or the distance between the image features of the two images, etc.

[0050] Understandably, the image feature differences are calculated globally for two adjacent stable frames, which consumes very little computing power and has minimal impact on the time consumption of page-turning detection.

[0051] The first threshold is a pre-set numerical limit used to characterize significant changes in the image. This threshold can be adaptively set according to the specific size of the physical book, camera resolution, or lighting environment.

[0052] If the difference in image features between the current stable image and the previous stable image is greater than or equal to a first threshold, it means that the content of the current stable image has changed significantly compared to the previous stable image. In this case, there is a high probability that a page turn has occurred between the previous and current stable images. However, this global difference in image features could be caused by a genuine page-turning action, or it could be due to the user moving the physical book, or the electronic device shaking.

[0053] Therefore, if the difference in image features between the current stable image and the previous stable image is greater than or equal to the first threshold, it is necessary to further determine the first change region for the current stable image and the second change region for the previous stable image, and then determine the page-turning detection result of the current stable image based on the first change region and the second change region. For example, the difference in hash values ​​between the previous stable image and the current stable image can be calculated using a hash algorithm, and this difference can be used as the image feature difference. Based on this, if the image feature difference is greater than a first threshold, it indicates that the global picture of the two stable images has changed significantly. In this case, the motion image between the two stable images can be used as a reference image to locate the first change region of the current stable image and the second change region of the previous stable image. Then, page turning detection can be performed by combining the local changes reflected by the first and second change regions.

[0054] In this embodiment of the invention, before extracting the first and second change regions, an image feature difference comparison based on a first threshold is introduced as a coarse screening condition for page-turning detection, preserving the practical advantage of low computational consumption for global feature comparison. Furthermore, only when the image feature difference is greater than or equal to the first threshold, i.e., there is a high probability of page-turning, is the subsequent computationally intensive change region localization and refined page-turning detection process based on the change region triggered. This achieves a reasonable allocation of computational power, balancing the operational efficiency of page-turning detection with the final judgment accuracy.

[0055] Based on any of the above embodiments, the page turning detection method further includes: If the difference in image features between the stable image and the previous stable image is less than the first threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0056] Specifically, if the difference in image features between the current stable image and the previous stable image is less than the first threshold, it means that the current stable image and the previous stable image maintain consistency in global macroscopic features. The subtle differences between the current stable image and the previous stable image may be due to slight changes in ambient light, minor vibrations of the device, or minor disturbances around the main body of the page that do not change the image noise, rather than originating from the page-turning action.

[0057] Therefore, if the difference in image features between the current stable image and the previous stable image is less than the first threshold, the page turning detection result of the current stable image can be directly determined as no page turning, without having to perform steps 120 and 130 again.

[0058] For example, the difference in hash values ​​between the previous stable image and the current stable image can be calculated using a hash algorithm, and this difference can be used as the image feature difference. Based on this, if the image feature difference is less than a first threshold, it indicates that the motion image between the previous and current stable images may reflect actions that do not substantially affect the image acquisition of the physical book pages, such as a user waving their arm, a mosquito flying by, or environmental changes. After such actions are completed, the main subject of the image remains unchanged, so the page-turning detection result for pages not yet turned can be directly output.

[0059] In this embodiment of the invention, when the difference in image features is less than a first threshold, the page-turning detection result is directly determined as no page turn. This can promptly block and terminate the subsequent process of extracting and comparing changed regions. In most stable reading scenarios without page turns, this reduces the computing power and resource consumption of electronic devices, ensuring low-power operation of electronic devices and significantly accelerating the overall response speed of page-turning detection.

[0060] Based on any of the above embodiments, in step 130, determining the page-turning detection result of the stable image based on the first change region and the second change region includes: Determine the regional feature differences between the first and second change regions; If the difference in regional features is greater than or equal to the second threshold, the page-turning detection result of the stable image is determined to be page-turned; If the difference in regional features is less than the second threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0061] Specifically, after determining the first and second change regions, the actual image content contained in the first and second change regions can be compared to obtain the regional feature differences between them. Here, the regional feature difference refers to the degree of feature difference or pixel difference between the image content of the current stable image in the first change region and the image content of the previous stable image in the second change region.

[0062] In some embodiments, regional feature differences can be calculated using local feature matching algorithms. For example, a feature point extraction algorithm can be used to extract a first set of feature points within a first region of change and a second set of feature points within a second region of change, and then the feature description distance between the first and second sets of feature points can be calculated as a quantified value of the regional feature difference. It is understood that the larger the value of the regional feature difference, the more significant the difference in the underlying image texture between the first and second regions of change.

[0063] The feature point extraction algorithm can be SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), or ORB (Oriented FAST and Rotated BRIEF), etc. The feature description distance can be Euclidean distance or Hamming distance, etc.

[0064] For example, Stable images acquired at different times are , The previous stable image is Stable images acquired over time . The set of feature points in the first region of change is , The set of feature points in the second region of change is Therefore, the regional characteristic differences can be calculated. , It indicates the position of a pixel in the image.

[0065] After calculating the regional feature differences, these differences can be compared with a second threshold. Here, the second threshold is a pre-defined numerical limit used to characterize a substantial replacement of the page content.

[0066] Specifically, if the difference in regional features is less than a second threshold, the page-turning detection result of the current stable image is determined to be "no page turned." For example, suppose the user only briefly moves the physical book out of the camera's capture area and then puts it back on the same page, or simply covers the page with their hand. During this process, although the image undergoes drastic changes, since it is essentially still the same page, the local features of the page pattern, text layout, etc., within the first and second changed areas remain highly consistent. In this case, the calculated difference in regional features is less than the second threshold, thus accurately identifying the page identity, determining that no page turning has occurred, and continuing the current reading progress.

[0067] If the difference in regional features is greater than or equal to a second threshold, the page-turning detection result of the current stable image is determined to be a page turned. For example, suppose a user actually turns to a new page in a book. Since the content and text layout of the new page may differ from the previous page, even if the first and second change regions define the same perturbation space, the extracted local feature points will not match at all. In this case, the calculated difference in regional features will increase sharply. When the difference in regional features is greater than or equal to the second threshold, it can be determined that the content of the image in the changed region has been substantially replaced, thus determining that a page-turning action has occurred and triggering the reading progress.

[0068] In this embodiment of the invention, by extracting the regional feature differences between the first and second change regions and comparing them with a second threshold, the two different scenarios of same-page reset interference and real new page opening are effectively distinguished, which greatly reduces the misjudgment rate in complex interaction scenarios and further improves the accuracy and reliability of page turning detection.

[0069] Based on any of the above embodiments, step 110, identifying each frame of the image sequence to be detected as a stable image or a moving image, includes: Detect the image change state between every two adjacent frames in the image sequence; Based on the image change state between every two adjacent frames, each frame is determined to be either a stable image or a moving image.

[0070] Specifically, two adjacent images refer to two consecutive images arranged in chronological order within the image sequence to be detected, for example... Images captured in real time and Images captured in real time That is, two adjacent frames.

[0071] In an image sequence, for each pair of adjacent images, the changes in the images between them can be detected. Here, the changes in the images can refer to whether there have been any changes between the two adjacent images, or it can be understood as whether the two adjacent images are visually consistent.

[0072] Image changes can be determined by comparing two adjacent frames. This comparison can be performed using global pixel difference calculations, structural similarity calculations, or calculations of the difference in global hash values. This allows for accurate determination of whether there are any real changes in the image content between the two adjacent frames, thus obtaining the accurate image change status.

[0073] After obtaining the image change state between every two adjacent frames in the image sequence, it is possible to determine whether each frame in the image sequence is a stable image or a moving image.

[0074] It is understandable that each frame in an image sequence is acquired sequentially over time, and whether each frame is stable or in motion can be determined by the changes in the images of its adjacent frames.

[0075] Specifically, if the image change indicates a change between two adjacent frames, it means that there is action happening in the picture. Thus, the later image in the two adjacent images can be identified as the moving image, which represents the dynamic process of the page being continuously turned or disturbed.

[0076] Conversely, if the image change indicates that there is no change between two adjacent frames, it means that the content of the image is in a relatively static state, and such unchanged images can be identified as stable images by following the timeline.

[0077] In this embodiment of the invention, the traditional single-frame static recognition is upgraded to inter-frame dynamic temporal analysis by detecting image change states. This uses the image change states between adjacent images as the basis for judgment, providing conditions for identifying each image as a moving image or a stable image according to the temporal sequence.

[0078] In some embodiments, to ensure the accuracy of image change state detection, smoothing processing can be performed on each frame of the image sequence beforehand to eliminate random noise in the image. This smoothing processing can be implemented using image denoising algorithms such as Gaussian smoothing, mean filtering, or bilateral filtering.

[0079] Based on any of the above embodiments, step 110, determining whether each frame image is a stable image or a moving image based on the image change state between every two adjacent frames, includes: For each frame of an image, a preset number of consecutive adjacent images are determined with the last frame of that image. If there is no change in the image state between consecutive adjacent images in the preset number of groups, the image is determined to be a stable image.

[0080] Specifically, using an image as the last frame means taking the currently being detected frame as the end of the time window on the timeline and tracing back towards historical time. The predetermined number of consecutive adjacent images refers to a predetermined number of frames that are closely connected in time. The predetermined number can also be understood as a pre-set number of inter-frame comparisons. This predetermined number can be an integer greater than or equal to 2; in some embodiments, the threshold for the predetermined number can be 3, 4, or 5.

[0081] For example, assuming the preset number is 3, for the currently being detected number 1... Frame image, can be the first Frame and the The frame image is the first group, the second... Frame and the Frame 2, the second Frame and the The frame is designated as the third group. These three groups of images constitute the frame numbered 3. A frame is a preset number of consecutive adjacent images of the last frame.

[0082] After acquiring the aforementioned preset number of consecutive adjacent images, the image change states of the aforementioned preset number of adjacent images can be integrated to identify the image as a stable image or a moving image.

[0083] For example, in a preset number of adjacent images, if the image state of any one of the groups shows a change, it is determined that the image to be detected is still in continuous flipping or has been subjected to continuous external interference. In this case, the image to be detected is still identified as a moving image, and the electronic device will continue to monitor subsequent consecutive frames. Only when the image state of each group of adjacent images in the preset number of adjacent images shows no change is it determined that all dynamic interference in the picture has been completely eliminated, and the current page has entered a truly stable state, and the image to be detected is determined as a stable image.

[0084] In this embodiment of the invention, a preset number of consecutive adjacent images are introduced to construct a window in the time dimension that can achieve image stabilization and filtering. Based on the image change state between the preset number of consecutive adjacent images, it is possible to identify whether the image is a stable image, thereby achieving accurate distinction between motion state and stable state. This ensures the reliability of subsequent page-turning detection based on stable images and greatly reduces the false judgment rate of page-turning detection.

[0085] Based on any of the above embodiments, this invention provides a page-turning detection method, which is applied to a picture book reading robot and can realize page-turning detection for picture books. The method includes the following steps: First, acquire image sequences.

[0086] Specifically, the camera on the picture book reading robot can capture images of picture book pages at preset intervals, thus forming an image sequence to be detected. In addition, to eliminate interference from noise, the picture book reading robot can smooth each captured image frame.

[0087] Subsequently, each frame in the image sequence to be detected is identified as either a stable image or a moving image.

[0088] Specifically, the picture book reading robot can detect the image change state between every two adjacent images in an image sequence. Based on this, for each image, the picture book reading robot takes that image as the last frame and traces back 3-5 consecutive adjacent images. If there is no change in the image change state between the 3-5 consecutive adjacent images, the page is determined to have entered a stable state, and the image is identified as a stable image; if a change is detected between the 3-5 consecutive adjacent images, the image is identified as a moving image.

[0089] Next, a first-level coarse screening check is performed on each stable image frame.

[0090] Specifically, after acquiring the current stable image, the picture book reading robot retrieves the previous stable image from the image sequence. From this, it can calculate the image feature difference between the current stable image and the previous stable image, and determine whether this image feature difference is greater than or equal to a first threshold. If the difference in image features is less than the first threshold, it means that the global macroscopic features of the two stable images have hardly changed. The picture book reading robot directly determines that the page turning detection result of the current stable image is not turned, and the page turning detection for the current stable image ends. If the difference in image features is greater than or equal to the first threshold, it indicates that there has been a macroscopic change in the image that appears to be a page turning, and then the second level of fine detection is triggered.

[0091] Subsequently, for cases where the image feature difference is greater than or equal to the first threshold, a second level of fine detection is performed.

[0092] Specifically, when the difference in image features is greater than or equal to a first threshold, the picture book reading robot can select a motion image representing the stage of the action as a reference image between the previous stable image and the current stable image. Thus, the picture book reading robot can compare the current stable image with the reference image to identify a first region of change in the stable image compared to the reference image; and compare the previous stable image with the same reference image to identify a second region of change in the previous stable image compared to the reference image.

[0093] Subsequently, the picture book reading robot can use a local feature matching algorithm to extract the underlying image texture feature points in the first and second change regions, and calculate the difference between the two, thereby determining the regional feature differences between the first and second change regions.

[0094] If the difference in regional features is less than the second threshold, it means that the image content of the region where the action interference occurred has not changed in essence before and after the action, and it is determined to be a page reset. At this time, it can be determined that the page turning detection result of the current stable image is no page turning. If the difference in regional features is greater than or equal to the second threshold, it indicates that the underlying image content of the disturbed region has been replaced, and it is determined to be a real page-turning behavior. At this time, the page-turning detection result of the current stable image is determined to be a page-turned image.

[0095] In this embodiment of the invention, by recognizing stable images, performing a first-level coarse screening and verification, and a second-level fine detection, the invention effectively solves the problem of misjudgment caused by non-page-turning interference while retaining the advantage of low computational power consumption of traditional algorithms. It achieves identity verification in the same-page reset scenario and ensures a balance between page-turning detection accuracy and operating efficiency.

[0096] The page-turning detection device provided by the present invention is described below. The page-turning detection device described below can be referred to in correspondence with the page-turning detection method described above.

[0097] Figure 2 This is a schematic diagram of the page-turning detection device provided by the present invention, as shown below. Figure 2 As shown, the device includes: State recognition unit 210 is used to identify whether each frame of the image sequence to be detected is a stable image or a moving image; The feature extraction unit 220 is used to determine, for each frame of stable image, a first change region of the stable image compared to a reference image, and a second change region of the previous stable image of the stable image compared to the reference image; the reference image is a motion image arranged between the stable image and the previous stable image in the image sequence. The page-turning detection unit 230 is used to determine the page-turning detection result of the stable image based on the first change region and the second change region.

[0098] The apparatus provided in this invention introduces a motion image in an intermediate transition state as a reference image, thereby determining a first change region between the stable image and the reference image, and a second change region between the previous stable image and the reference image, achieving precise positioning of the image change region before and after motion interference. Therefore, page-turning detection is performed based on the first and second change regions, reducing the traditional blind comparison of the entire frame to a precise comparison of the image change region before and after motion interference. This effectively filters out various instantaneous interference factors affecting page-turning detection, achieves page identity verification, avoids misjudgment due to page resetting, and significantly improves the anti-interference capability, robustness, and accuracy of page-turning detection.

[0099] Based on any of the above embodiments, the feature extraction unit is specifically used for: If the difference in image features between the stable image and the previous stable image is greater than or equal to a first threshold, a first change region of the stable image compared to the reference image and a second change region of the previous stable image compared to the reference image are determined.

[0100] Based on any of the above embodiments, the page-turning detection unit is further configured to: If the difference in image features between the stable image and the previous stable image is less than the first threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0101] Based on any of the above embodiments, the page-turning detection unit is specifically used for: Determine the regional feature differences between the first and second change regions; If the difference in regional features is greater than or equal to the second threshold, the page-turning detection result of the stable image is determined to be page-turned; If the difference in regional features is less than the second threshold, the page-turning detection result of the stable image is determined to be no page turn.

[0102] Based on any of the above embodiments, the state recognition unit is specifically used for: Detect the image change state between every two adjacent frames in the image sequence; Based on the image change state between every two adjacent frames, each frame is determined to be either a stable image or a moving image.

[0103] Based on any of the above embodiments, the state recognition unit is specifically used for: For each frame of an image, a preset number of consecutive adjacent images are determined with the last frame of that image. If there is no change in the image state between consecutive adjacent images in the preset number of groups, the image is determined to be a stable image.

[0104] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a page-turning detection method, which includes: Identify whether each frame in the image sequence to be detected is a stable image or a moving image; For each frame of a stable image, a first region of change in the stable image compared to a reference image is determined, and a second region of change in the previous stable image of the stable image compared to the reference image is determined; the reference image is a motion image arranged in the image sequence between the stable image and the previous stable image. Based on the first change region and the second change region, the page-turning detection result of the stable image is determined.

[0105] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer being able to execute the page-turning detection method provided by the above methods, the method comprising: Identify whether each frame in the image sequence to be detected is a stable image or a moving image; For each frame of a stable image, a first region of change in the stable image compared to a reference image is determined, and a second region of change in the previous stable image of the stable image compared to the reference image is determined; the reference image is a motion image arranged in the image sequence between the stable image and the previous stable image. Based on the first change region and the second change region, the page-turning detection result of the stable image is determined.

[0107] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the page-turning detection method provided by the methods described above, the method comprising: Identify whether each frame in the image sequence to be detected is a stable image or a moving image; For each frame of a stable image, a first region of change in the stable image compared to a reference image is determined, and a second region of change in the previous stable image of the stable image compared to the reference image is determined; the reference image is a motion image arranged in the image sequence between the stable image and the previous stable image. Based on the first change region and the second change region, the page-turning detection result of the stable image is determined.

[0108] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A page-turning detection method, characterized in that, include: Identify whether each frame in the image sequence to be detected is a stable image or a moving image; For each frame of a stable image, a first region of change in the stable image compared to a reference image is determined, and a second region of change in the previous stable image of the stable image compared to the reference image is determined. The reference image is a moving image arranged in the image sequence between the stable image and the previous stable image; Based on the first change region and the second change region, the page-turning detection result of the stable image is determined.

2. The page-turning detection method according to claim 1, characterized in that, Determining the first change region of the stable image relative to the reference image, and the second change region of the previous stable image of the stable image relative to the reference image, includes: If the difference in image features between the stable image and the previous stable image is greater than or equal to a first threshold, a first change region of the stable image compared to the reference image and a second change region of the previous stable image compared to the reference image are determined.

3. The page-turning detection method according to claim 2, characterized in that, Also includes: If the difference in image features between the stable image and the previous stable image is less than the first threshold, the page-turning detection result of the stable image is determined to be no page turn.

4. The page-turning detection method according to any one of claims 1 to 3, characterized in that, The step of determining the page-turning detection result of the stable image based on the first change region and the second change region includes: Determine the regional feature differences between the first and second change regions; If the difference in regional features is greater than or equal to the second threshold, the page-turning detection result of the stable image is determined to be page-turned; If the difference in regional features is less than the second threshold, the page-turning detection result of the stable image is determined to be no page turn.

5. The page-turning detection method according to any one of claims 1 to 3, characterized in that, The identification of each frame in the image sequence to be detected as a stable image or a moving image includes: Detect the image change state between every two adjacent frames in the image sequence; Based on the image change state between every two adjacent frames, each frame is determined to be either a stable image or a moving image.

6. The page-turning detection method according to claim 5, characterized in that, The step of determining whether each frame of an image is a stable image or a moving image based on the image change state between every two adjacent frames includes: For each frame of an image, a preset number of consecutive adjacent images are determined with the last frame of that image. If there is no change in the image state between consecutive adjacent images in the preset number of groups, the image is determined to be a stable image.

7. A page-turning detection device, characterized in that, include: The state recognition unit is used to identify whether each frame in the image sequence to be detected is a stable image or a moving image; The feature extraction unit is used to determine, for each frame of stable image, a first change region of the stable image compared to a reference image, and a second change region of the previous stable image of the stable image compared to the reference image. The reference image is a moving image arranged in the image sequence between the stable image and the previous stable image; The page-turning detection unit is used to determine the page-turning detection result of the stable image based on the first change region and the second change region.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the page-turning detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the page-turning detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the page-turning detection method as described in any one of claims 1 to 6.

Images