Image processing method and terminal, device, electronic equipment, storage medium

Abstract

Embodiments of the present application provide an image processing method and terminal, device, electronic equipment and storage medium, wherein the method comprises: performing a reduction processing on a first image in a sequence of image frames to obtain a second image; performing position recognition on an indication object in the second image to obtain an indication position of the indication object in the second image; updating a current selection position based on the indication position to obtain an updated selection position, the current selection position being determined based on an indication position corresponding to a previous frame image of the first image; selecting a third image corresponding to a target position from the first image, the target position comprising one of the current selection position and the updated selection position; and performing recognition on a target object in the third image to obtain a recognition result. The method of the present application can reduce the overall operation resource consumption and overall time consumption required for recognizing a target object indicated by an indication object, and improve the overall recognition efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer vision technology, and more specifically, the embodiments of the present invention relate to an image processing method and terminal, device, electronic device, and storage medium. Background Technology

[0002] This section is intended to provide background or context for embodiments of the invention as set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] With the development of computer vision technology, there are increasingly more applications using image processing techniques to identify target objects. In some applications, image processing technology can be used to identify the target object pointed to by a pointer (such as a finger, a dictionary pen, etc.). For example, in a fingertip word lookup scenario, a user can point to a word to be identified on a reading material with their finger. Using image processing technology, the word pointed to by the user's finger can be identified, thereby assisting the user in reading or learning new words.

[0004] However, in related technologies, the schemes for identifying the target object indicated by the pointer have a large overall computational resource overhead, are time-consuming, and have low recognition efficiency. Summary of the Invention

[0005] Therefore, there is a great need for an improved image processing method and related products to reduce the computational resource overhead and time consumption required to identify the target object indicated by the pointer.

[0006] In this context, embodiments of the present invention are intended to provide an image processing method and terminal, apparatus, electronic device, and storage medium.

[0007] In a first aspect of the present invention, an image processing method is provided, applied to a terminal, comprising:

[0008] The first image in the image frame sequence is scaled down to obtain the second image;

[0009] The location of the indicated object in the second image is identified to obtain the indicated position of the indicated object in the second image;

[0010] The current selected position is updated based on the indicated position to obtain the updated selected position; wherein the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image;

[0011] A third image corresponding to the target location is selected from the first image, wherein the target location includes one of the following: the current selected location, or the updated selected location;

[0012] The target object in the third image is identified, and the identification result is obtained.

[0013] In some embodiments of the present invention, the method further includes: determining the movement state of the indicated object based on the indicated position and the historical indicated position corresponding to at least one frame in the image frame sequence preceding the first image; displaying the second image when the movement state indicates that the indicated object has stopped moving; and / or outputting the recognition result according to a set output method; the output method includes at least one of the following: display output, voice output. This allows for rapid output of the recognition result when it is determined that the indicated object has stopped moving, reducing the user's waiting time.

[0014] In other embodiments of the present invention, updating the current selected position based on the indicated position to obtain an updated selected position includes: if the indicated position is different from the current selected position, determining the indicated position as the updated selected position. This allows for timely updates to the selected position when the indicated position changes, thereby further improving the accuracy of target object identification.

[0015] In some embodiments of the present invention, the terminal includes a processor and peripheral devices communicatively connected to the processor, the peripheral devices including an image processing module; the step of reducing the size of a first image in an image frame sequence to obtain a second image includes: the image processing module reducing the size of the first image in the image frame sequence to obtain a second image; the step of identifying the location of an indicator object in the second image to obtain the indicator position of the indicator object in the second image includes: the processor identifying the location of the indicator object in the second image to obtain the indicator position of the indicator object in the second image; the step of updating the current selected position based on the indicator position to obtain an updated selected position includes: the processor updating the current selected position based on the indicator position to obtain an updated selected position; the step of selecting a third image corresponding to a target position from the first image includes: the image processing module selecting a third image corresponding to a target position from the first image; the step of identifying a target object in the third image to obtain an identification result includes: the processor identifying the target object in the third image to obtain an identification result. In this way, on the one hand, since the image processing module performs both the reduction processing of the first image and the selection of the third image corresponding to the target position from the first image, the processor overhead can be reduced, and the efficiency of image reduction and selection processing can be improved, thereby further improving the overall efficiency of target object recognition. On the other hand, since the second and third images processed by the processor do not require processing of the complete first image, the amount of data copied to memory during processor processing can be reduced, the processor's data throughput can be reduced, and the overall computing resource overhead and time consumption required for target object recognition can be further reduced, thereby improving the processor's operating performance.

[0016] In some embodiments of the present invention, the method further includes at least one of the following: the image processing module transmits the second image to the processor using a zero-copy method; the image processing module transmits the third image to the processor using a zero-copy method. This further reduces the amount of data copied to memory during processor processing, thereby further reducing the processor's data throughput and improving its operating performance.

[0017] In some embodiments of the present invention, the image processing module includes an image reduction component and an image cropping component; the image processing module reduces the size of a first image in an image frame sequence to obtain a second image, including: the image reduction component reduces the size of the first image in the image frame sequence to obtain a second image; the image processing module selects a third image corresponding to a target position from the first image, including: the image cropping component crops the third image corresponding to the target position from the first image. Thus, the first image can be processed in parallel by the image reduction component and the image cropping component to obtain the second and third images respectively, thereby further improving the overall efficiency of target object recognition.

[0018] In some embodiments of the present invention, the peripheral device further includes an image acquisition module and an image transmission module; the method further includes: the image acquisition module acquiring the first image; and the image transmission module transmitting the first image to the image reduction component and the image cropping component respectively through the same data output channel. This reduces data transmission costs.

[0019] In a second aspect of the present invention, a terminal is provided, comprising: a processor and peripheral devices communicatively connected to the processor, the peripheral devices including an image processing module; wherein the image processing module is configured to: reduce the size of a first image in an image frame sequence to obtain a second image; select a third image corresponding to a target position from the first image; the processor is configured to: perform position recognition on an indicated object in the second image to obtain an indicated position of the indicated object in the second image; update the currently selected position based on the indicated position to obtain an updated selected position; and recognize a target object in the third image to obtain a recognition result; wherein the target position includes one of the following: the currently selected position, the updated selected position.

[0020] In some embodiments of the present invention, the processor includes a central processing unit (CPU) and a neural network processor. The neural network processor is configured to: perform position recognition on an indicated object in the second image to obtain the indicated position of the indicated object in the second image; update the currently selected position based on the indicated position to obtain an updated selected position; recognize a target object in the third image to obtain a recognition result, and transmit the recognition result to the CPU; the CPU is configured to output the recognition result according to a set output method; the output method includes at least one of the following: display output and voice output. In this way, the high processing performance of the neural network processor can be utilized to improve the efficiency of position recognition and target object recognition, thereby further improving the overall efficiency of target object recognition.

[0021] In a third aspect of the present invention, an image processing apparatus is provided, comprising: a reduction module for reducing the size of a first image in an image frame sequence to obtain a second image; a first recognition module for recognizing the position of an indicator object in the second image to obtain an indicator position of the indicator object in the second image; an update module for updating a currently selected position based on the indicator position to obtain an updated selected position; wherein the current selected position is determined based on the indicator position corresponding to the previous frame image of the first image; a selection module for selecting a third image corresponding to a target position from the first image, the target position including one of the following: the current selected position, the updated selected position; and a second recognition module for recognizing a target object in the third image to obtain a recognition result.

[0022] In a fourth aspect of the present invention, a terminal is provided, including a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement some or all of the steps in the method described in any of the above embodiments.

[0023] In a fifth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the method described in any of the above embodiments.

[0024] According to the image processing method, terminal, apparatus, electronic device, and storage medium of the present invention, in the process of recognizing the target object indicated by an indicator object in a first image, the first image can be scaled and selected to obtain a second image and a third image. The position of the indicator object in the second image is obtained by position recognition, and the recognition result is obtained by recognizing the target object in the third image. Thus, on the one hand, since the second image is a scaled-down version of the first image, the original first image does not need to be processed during the position recognition of the indicator object, thereby reducing the computational resource overhead and time consumption required for position recognition and improving the efficiency of position recognition. On the other hand, since the third image is a selected portion of the first image, the complete first image does not need to be processed during the recognition of the target object in the third image, thereby reducing the computational resource overhead and time consumption required for target object recognition and improving the efficiency of target object recognition. Therefore, the overall computational resource overhead and overall time consumption required for recognizing the target object indicated by the indicator object can be reduced, improving the overall recognition efficiency and providing a better user experience. Furthermore, the third image is selected from the first image based on the target location. The target location can include the current selection location (i.e., the selection location before the update) or the selection location updated based on the indicator location of the indicator object in the second image. When the target location includes the selection location before the update, since the selection location before the update is determined based on the indicator location corresponding to the previous frame of the first image in the image frame sequence, the operations of selecting the third image from the first image and identifying the target object in the third image do not need to rely on the result of position recognition of the indicator object in the second image, thereby further improving the overall recognition efficiency. When the target location includes the updated selection location, since the updated selection location is updated based on the indicator location of the indicator object in the second image, the accuracy of target object recognition can be further improved. Attached Figure Description

[0025] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein:

[0026] Figure 1A The diagram schematically illustrates an application scenario for identifying a target object indicated by an indicator object, as provided by an embodiment of the present invention.

[0027] Figure 1B A flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown schematically.

[0028] Figure 2 A flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown schematically.

[0029] Figure 3A A schematic block diagram illustrating the structural composition of a terminal according to an embodiment of the present invention is shown.

[0030] Figure 3B A schematic block diagram illustrating the structural composition of a terminal according to an embodiment of the present invention is shown.

[0031] Figure 3C A schematic block diagram illustrating the structural composition of a terminal according to an embodiment of the present invention is shown.

[0032] Figure 3D A schematic block diagram illustrating the structural composition of a terminal according to an embodiment of the present invention is shown.

[0033] Figure 4 A schematic block diagram illustrating the composition of an image processing apparatus according to an embodiment of the present invention is shown.

[0034] Figure 5A This diagram illustrates an embodiment of the present invention that performs reduction and cropping processing on images captured by a camera.

[0035] Figure 5B A flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown schematically.

[0036] Figure 5C A schematic diagram illustrating the implementation architecture of an image processing method according to an embodiment of the present invention is shown.

[0037] Figure 5D A data flow diagram of an image processing method according to an embodiment of the present invention is shown schematically;

[0038] Figure 6 A schematic diagram of the hardware entity of an electronic device according to an embodiment of the present invention is shown.

[0039] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0040] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0041] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first, second, third" used in the following description are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permissible, so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments and implementations of the invention only and is not intended to limit this disclosure. Invention Overview

[0044] The inventors have discovered that the scheme for identifying the target object indicated by the indicator in the related technology is usually implemented in the following way: First, an image containing the indicator and the target object is acquired; then, the position of the indicator in the acquired original image is identified to obtain the indicator position of the indicator in the original image; finally, based on the indicator position of the indicator in the original image, the target object in the original image is identified to obtain the identification result.

[0045] In the above-mentioned related technical solutions, the identification of the indicated location and the identification of the target object are both performed on the acquired original image. Since the acquired original image is usually high in clarity and large in image data, the computational resources required for the location identification and target object identification on the original image are large and time-consuming. Consequently, the overall computational resource consumption is large, the time consumption is long, and the identification efficiency is low during the process of identifying the target object indicated by the indicated object.

[0046] In some related technical solutions, the operations of location recognition and target object recognition on the original image are both performed by the central processing unit (CPU) in the terminal. During the CPU operation, the data stream needs to be transmitted through memory copying. Since the data volume of the original image is usually large, the data stream transmitted during the location recognition and target object recognition operations will occupy a lot of memory bandwidth, which will slow down the overall processing performance of the CPU, thereby further increasing the overall time spent on recognizing the target object indicated by the pointer and reducing the overall recognition efficiency.

[0047] In view of this, embodiments of the present invention provide at least one image processing method, terminal, device, electronic device, and storage medium. According to embodiments of the present invention, in the process of recognizing the target object indicated by an indicator in a first image, the first image can be scaled and selected to obtain a second image and a third image. The position of the indicator in the second image is obtained by recognizing the position of the indicator in the second image, and the recognition result is obtained by recognizing the target object in the third image. Thus, on the one hand, since the second image is a scaled-down version of the first image, the original first image does not need to be processed during the position recognition of the indicator, thereby reducing the computational resource overhead and time consumption required for position recognition and improving the efficiency of position recognition. On the other hand, since the third image is a selected portion of the first image, the complete first image does not need to be processed during the recognition of the target object in the third image, thereby reducing the computational resource overhead and time consumption required for target object recognition and improving the efficiency of target object recognition. Therefore, the overall computational resource overhead and overall time consumption required for recognizing the target object indicated by the indicator can be reduced, improving the overall recognition efficiency and providing a better user experience. Furthermore, the third image is selected from the first image based on the target location. The target location can include the current selection location (i.e., the selection location before the update) or the selection location updated based on the indicator location of the indicator object in the second image. When the target location includes the selection location before the update, since the selection location before the update is determined based on the indicator location corresponding to the previous frame of the first image in the image frame sequence, the operations of selecting the third image from the first image and identifying the target object in the third image do not need to rely on the result of position recognition of the indicator object in the second image, thereby further improving the overall recognition efficiency. When the target location includes the updated selection location, since the updated selection location is updated based on the indicator location of the indicator object in the second image, the accuracy of target object recognition can be further improved.

[0048] After introducing the basic principles of the present invention, various non-limiting embodiments of the present invention will be described in detail below.

[0049] Application Scenarios Overview

[0050] First refer to Figure 1A , Figure 1A This diagram schematically illustrates an application scenario for identifying a target object indicated by an indicator, as provided by an embodiment of the present invention. For example... Figure 1AAs shown, terminal 10 may have a camera (not shown) and a display screen 11. The user can point the pointer 20 at the target object 30 to be identified. Terminal 10 can use the camera to capture images of the area containing the pointer 20 and the target object 30, and identify the target object 30 indicated by the pointer 20 based on the first image captured by the camera, and display the identification result on the display screen 11.

[0051] During the process of identifying the target object indicated by the indicator object in the first image, the terminal may perform the following operations: reduce the size of the first image in the image frame sequence to obtain a second image; identify the position of the indicator object in the second image to obtain the indicator position of the indicator object in the second image; update the current selected position based on the indicator position to obtain an updated selected position; wherein the current selected position is determined based on the indicator position corresponding to the previous frame image of the first image; select a third image corresponding to the target position from the first image, wherein the target position includes one of the following: the current selected position or the updated selected position; identify the target object in the third image to obtain an identification result.

[0052] It should be noted that the aforementioned terminals can be various electronic devices, including but not limited to at least one of the following: smart desk lamps, smartphones, electronic dictionaries, smart learning machines, tablets, laptops, desktop computers, smart TVs, smart fans, portable video players, personal digital assistants, and portable gaming devices.

[0053] Exemplary methods

[0054] The following is combined Figure 1A Application scenarios, refer to Figure 1B This description outlines an image processing method for recognizing a target object indicated by an indicator, according to an exemplary embodiment of the present invention. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the invention, and the embodiments of the invention are not limited in any way. Rather, the embodiments of the invention can be applied to any applicable scenario.

[0055] This invention provides an image processing method that can be executed by a terminal. Figure 1B The flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown schematically. Figure 1B As shown, the method includes the following steps S101 to S105:

[0056] Step S101: Reduce the size of the first image in the image frame sequence to obtain the second image.

[0057] Here, the image frame sequence may include multiple consecutive frames from an image stream captured by a camera, or it may include multiple frames taken from the image stream at set time intervals. In practice, the image frame sequence may be captured by the terminal's camera, or it may be captured by a camera or video camera outside the terminal, etc., and this embodiment of the invention is not limited in this respect.

[0058] The first image is the image to be processed in the image frame sequence. In practice, the terminal can process each frame in the image frame sequence sequentially.

[0059] The second image is obtained by reducing the size of the first image. The reduced-size first image retains the positional information of the indicated object while reducing the number of pixels, thus reducing the amount of image data. In practice, the first image can be reduced to a set size, or it can be reduced proportionally according to a set reduction ratio; there is no limitation on this.

[0060] Step S102: Perform position recognition on the indicator object in the second image to obtain the indicator position of the indicator object in the second image.

[0061] Here, the object of instruction can be any suitable object that can be used for instruction, including but not limited to at least one of the following: a finger, a reading pen, a pencil, a pen, and a pointer stick.

[0062] In implementation, any suitable object recognition algorithm can be used to identify the location of the indicated object in the second image; there are no limitations. For example, the object recognition algorithm may include, but is not limited to, at least one of Convolutional Neural Networks (CNN), Region-based CNN (R-CNN), and Fast R-CNN.

[0063] By identifying the location of the indicated object in the first image, the indicated object in the second image and its indicated location in the second image can be obtained. The indicated location may include the area indicated by the indicated object in the second image.

[0064] In some implementations, the indication location may include the location area where the indication object is located. For example, the indication location of the indication object in the second image may be at least one of a rectangular area, a circular area, an elliptical area, etc., containing a set size of the indication object. In practice, the indication object in the second image and its location in the second image can be obtained by performing position recognition on the indication object in the second image; based on the object location, the location area where the indication object is located can be determined, that is, the indication location of the indication object in the second image.

[0065] In some embodiments, the indication position may include a region adjacent to the indicated object along a specific indication direction. For example, the indication position of the indicated object in the second image may be at least one of a rectangular region, a circular region, an elliptical region, etc., of a predetermined size adjacent to the indicated object along a specific indication direction. In practice, the indication object and its position in the second image can be obtained by performing position recognition on the indicated object in the second image; based on the object position, the region adjacent to the indicated object along the specific indication direction, i.e., the indication position of the indicated object in the second image, can be determined. Here, the specific indication direction may be preset, or it may be determined based on the pose information of the indicated object in the second image after recognition of the pose information; the embodiments of the present invention are not limited in this respect.

[0066] Step S103: Update the current selected position based on the indicated position to obtain the updated selected position; wherein, the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image.

[0067] Here, the current selection position is determined based on the indicator position corresponding to the previous frame of the first image. In implementation, the previous frame image can be scaled down to obtain a scaled-down previous frame image; the position of the indicator object in the scaled-down previous frame image can be identified to obtain the indicator position of the indicator object in the scaled-down previous frame image, that is, the indicator position corresponding to the previous frame image; based on the indicator position corresponding to the previous frame image, the selection position corresponding to the previous frame image can be updated to obtain the current selection position corresponding to the first image.

[0068] In some implementations, the indicated location can be directly determined as the updated selected location.

[0069] In some implementations, if the indicated position differs from the currently selected position, the indicated position can be determined as the updated selected position. This allows for timely updates to the selected position when the indicated position changes, thereby further improving the accuracy of target object identification.

[0070] Step S104: Select a third image corresponding to the target position from the first image. The target position includes one of the following: the current selection position or the updated selection position.

[0071] Here, the target location can be the current selected location, or it can be the updated selected location obtained by updating the current selected location based on the indicated location.

[0072] In some implementations, the current selected position can be used as the target position if the current selected position has not been updated; if the current selected position has been updated, the updated selected position can be used as the target position.

[0073] Step S105: Identify the target object in the third image to obtain the identification result.

[0074] Here, the target object can be any suitable object to be identified, including but not limited to at least one of the following: text content (such as words or Chinese characters) in a reading material, image objects (such as animal pictures, plant pictures, or object pictures) in a reading material, or objects in a real-world scene (such as animals, plants, or objects). In implementation, any suitable object recognition algorithm can be used to identify the target object in the third image; there are no limitations. For example, the object recognition algorithm can include but is not limited to at least one of the following: CNN algorithm, R-CNN algorithm, and Fast R-CNN algorithm.

[0075] In some implementations, when the target object is the text content in a reading material, the recognition result may include, but is not limited to, at least one of the following: text information corresponding to the target object, text interpretation, etc. In practice, a text recognition algorithm (such as Optical Character Recognition (OCR) algorithm, R-CNN algorithm, etc.) can be used to recognize the text in the third image to obtain the recognition result.

[0076] In some implementations, when the target object is an image object in a reading material or an object in a real scene, the recognition result may include, but is not limited to, at least one of the target object's name, type, and associated knowledge.

[0077] It should be noted that, during implementation, the above steps S101 and S104 can be executed by the terminal's hardware module (such as an image processing module) or by the application (APP) running on the terminal; there is no limitation here.

[0078] In the embodiments of the present invention described above, on the one hand, since the second image is a scaled-down version of the first image, the original first image does not need to be processed during the location recognition process of the indicated object, thereby reducing the computational resource overhead and time consumption required for location recognition and improving the efficiency of location recognition. On the other hand, since the third image is a partial image selected from the first image, the complete first image does not need to be processed during the recognition of the target object in the third image, thereby reducing the computational resource overhead and time consumption required for target object recognition and improving the efficiency of target object recognition. Thus, the overall computational resource overhead and overall time consumption required for recognizing the target object indicated by the indicated object can be reduced, improving the overall recognition efficiency and providing a better user experience. Furthermore, the third image is selected from the first image based on the target position, which may include the current selection position (i.e., the selection position before the update) or the selection position updated based on the indicated position of the indicated object in the second image. When the target location includes the selected location before the update, since the selected location before the update is determined based on the indicator location corresponding to the previous frame of the first image in the image frame sequence, the operation of selecting the third image from the first image and recognizing the target object in the third image does not need to rely on the result of position recognition of the indicator object in the second image, thereby further improving the overall recognition efficiency; when the target location includes the selected location after the update, since the selected location after the update is updated based on the indicator location of the indicator object in the second image, the accuracy of target object recognition can be further improved.

[0079] In some embodiments of the present invention, the above method may further include the following steps S111 to S112:

[0080] Step S111: Based on the indicated position and the historical indicated position corresponding to at least one frame in the image frame sequence before the first image, determine the movement state of the indicated object.

[0081] Here, the movement state of the indicated object may include, but is not limited to, at least one of the following: a first state indicating that the indicated object has stopped moving, a second state indicating that the indicated object is moving.

[0082] In practice, the movement state of the indicated object can be determined by using an appropriate method based on the indicated position and the historical indicated position corresponding to at least one frame in the image frame sequence before the first image, depending on the actual situation. This is not limited here.

[0083] In some implementations, if the indicated position is the same as the historical indicated position corresponding to the previous N frames of the first image in the image frame sequence, the movement state of the indicated object can be determined as a first state indicating that the indicated object has stopped moving; if the indicated position is the same as the historical indicated position corresponding to any one of the previous N frames of the first image in the image frame sequence, the movement state of the indicated object can be determined as a second state indicating that the indicated object is moving; where N is a positive integer. Thus, when the indicated object stops moving, the current indicated position is at least the same as the indicated position corresponding to the previous frame image, and therefore the selected position before the update is the same as the selected position after the update. Therefore, regardless of whether the target position includes the selected position before the update or the selected position after the update, a relatively accurate recognition result can be obtained.

[0084] In some implementations, the movement state of the indicated object can be determined as a first state indicating that the indicated object has stopped moving when the offset between the indicated position and the historical indicated position corresponding to the previous N frames of the first image in the image frame sequence is less than a set offset threshold; and the movement state of the indicated object can be determined as a second state indicating that the indicated object is moving when the offset between the indicated position and the historical indicated position corresponding to any frame of the previous N frames of the first image in the image frame sequence is greater than or equal to the set offset threshold.

[0085] Step S112: When the movement state indicates that the indicated object has stopped moving, the second image is displayed, and / or the recognition result is output according to the set output method; the output method includes at least one of the following: display output, voice output.

[0086] This allows for rapid output of recognition results once the indicated object has stopped moving, reducing user waiting time.

[0087] In some embodiments of the present invention, the terminal includes a processor and peripheral devices communicatively connected to the processor, the peripheral devices including an image processing module. Figure 2 The flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown schematically. Figure 2 As shown, the method may include the following steps S201 to S205:

[0088] In step S201, the image processing module reduces the size of the first image in the image frame sequence to obtain the second image.

[0089] In step S202, the processor performs position recognition on the indicated object in the second image to obtain the indicated position of the indicated object in the second image.

[0090] In step S203, the processor updates the current selected position based on the indicated position to obtain the updated selected position; wherein, the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image.

[0091] Step S204: The image processing module selects a third image from the first image that corresponds to the target position; the target position includes one of the following: the current selection position or the updated selection position.

[0092] In step S205, the processor identifies the target object in the third image and obtains the identification result.

[0093] Here, the processor may include, but is not limited to, at least one of a CPU, a graphics processing unit (GPU), and a neural network processing unit (NPU). In some embodiments, an application program (APP) may run on the processor, and the processor may use the running APP to execute the above steps S202, S203, and / or S205.

[0094] The image processing module may include any suitable hardware module for processing images, including but not limited to at least one of an image signal processor (ISP) and a raster graphics accelerator (RGA).

[0095] In the above embodiments, on the one hand, since the processing of shrinking the first image and selecting the third image corresponding to the target position from the first image are both performed by the image processing module, the processor overhead can be reduced, and the efficiency of image shrinking and selection processing can be improved, thereby further improving the overall efficiency of target object recognition; on the other hand, since the second and third images processed by the processor do not require processing of the complete first image, the amount of data copied to memory during processor processing can be reduced, the processor's data throughput can be reduced, and the overall computing resource overhead and time consumption required for target object recognition can be further reduced, thereby improving the processor's operating performance.

[0096] In some embodiments of the present invention, the above method may further include at least one of the following steps S211 and S212:

[0097] In step S211, the image processing module transmits the second image to the processor using a zero-copy method.

[0098] In step S212, the image processing module transmits the third image to the processor using a zero-copy method.

[0099] Here, zero-copy refers to a method where no memory copying is required during data transfer.

[0100] In some implementations, the image processing module can output a second image / third image to a specific memory space, which the processor can access to obtain the second image / third image. Here, the specific memory space can be pre-defined or dynamically determined by the image processing module. If the memory space is dynamically determined by the image processing module, after outputting the second image / third image to that memory space, the image processing module can send the address of that memory space to the processor, allowing the processor to access the memory space based on that address. In practice, the same memory space can be used for transmitting the second image, and different memory spaces can be used for transmitting the third image; this embodiment of the invention is not limited in this respect.

[0101] In the above embodiments, the image processing module uses a zero-copy method to transmit the second image and / or the third image to the processor. This can further reduce the amount of data copied to memory during processor processing, thereby further reducing the processor's data throughput and improving the processor's operating performance.

[0102] In some embodiments of the present invention, the image processing module includes an image reduction component and an image cropping component; step S201 above may include the following step S221:

[0103] Step S221: The image downsizing component downscales the first image in the image frame sequence to obtain the second image;

[0104] Step S204 above may include the following step S222:

[0105] In step S222, the image cropping component crops out a third image corresponding to the target position from the first image.

[0106] Here, the image reduction component may include, but is not limited to, at least one of ISP and RGA; the image cropping component may also include, but is not limited to, at least one of ISP and RGA.

[0107] In the above embodiments, the first image can be processed in parallel by the image reduction component and the image cropping component to obtain the second image and the third image respectively, thereby further improving the overall efficiency of identifying the target object.

[0108] In some embodiments of the present invention, the peripheral device further includes an image acquisition module and an image transmission module; the above method may also include the following steps S231 to S232:

[0109] Step S231: The image acquisition module acquires the first image;

[0110] In step S232, the image transmission module transmits the first image to the image reduction component and the image cropping component respectively through the same data output channel.

[0111] In some implementations, the image acquisition module may be a camera module (such as an image sensor), and the image transmission module may include a CMOS Serial Interface (CSI) and / or an image processing controller, etc.

[0112] Exemplary terminal

[0113] After introducing the method of exemplary embodiments of the present invention, the following references are made. Figure 3A A terminal according to an exemplary embodiment of the present invention will be described.

[0114] Figure 3A The diagram schematically illustrates the structural composition of a terminal according to an embodiment of the present invention, such as... Figure 3A As shown, the terminal 10 includes: a processor 13 and peripheral devices 14 communicatively connected to the processor 13, the peripheral devices 14 including an image processing module 141; wherein,

[0115] Image processing module 141 is used to: reduce the size of a first image in an image frame sequence to obtain a second image; and select a third image corresponding to the target position from the first image.

[0116] Processor 13 is configured to: perform position recognition on an indicated object in a second image to obtain the indicated position of the indicated object in the second image; update the current selected position based on the indicated position to obtain the updated selected position; and recognize a target object in a third image to obtain a recognition result.

[0117] The target location includes one of the following: the current selected location or the updated selected location.

[0118] In some embodiments, the image processing module is also used for at least one of the following:

[0119] The second image is transferred to the processor using a zero-copy method;

[0120] The third image is transferred to the processor using a zero-copy method.

[0121] In some embodiments, the processor is further configured to: determine the movement state of the indicated object based on the indicated position and the historical indicated position corresponding to at least one frame image in the image frame sequence preceding the first image; display the second image when the movement state indicates that the indicated object has stopped moving; and / or output the recognition result according to a set output method; the output method includes at least one of the following: display output, voice output.

[0122] In some embodiments, see Figure 3B The image processing module 141 includes an image reduction component 141a and an image cropping component 141b; wherein, the image reduction component 141a is used to reduce the size of the first image in the image frame sequence to obtain the second image; the image cropping component 141b is used to crop out the third image corresponding to the target position from the first image.

[0123] In some embodiments, see Figure 3C The peripheral device 14 also includes an image acquisition module 142 and an image transmission module 143; wherein, the image acquisition module 142 is used to acquire a first image; the image transmission module 143 is used to transmit the first image to the image reduction component 141a and the image cropping component 141b respectively through the same data output channel.

[0124] In some embodiments, see Figure 3D The processor 13 includes a central processing unit 131 and a neural network processor 132; wherein, the neural network processor 132 is used to: perform position recognition on the indicated object in the second image to obtain the indicated position of the indicated object in the second image; update the current selected position based on the indicated position to obtain the updated selected position; recognize the target object in the third image to obtain the recognition result, and transmit the recognition result to the central processing unit 131; the central processing unit 131 is used to output the recognition result according to a set output mode; the output mode includes at least one of the following: display output, voice output.

[0125] The description of the terminal embodiments above is similar to that of the method embodiments above, and has similar beneficial effects. For technical details not disclosed in the device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.

[0126] Exemplary device

[0127] After introducing the method and terminal of exemplary embodiments of the present invention, the following references are made. Figure 4 An apparatus according to an exemplary embodiment of the present invention will be described.

[0128] Figure 4The schematic diagram illustrates the structural block diagram of an image processing apparatus according to an embodiment of the present invention, such as... Figure 4 As shown, the image processing device 400 includes: a reduction module 410, a first recognition module 420, an update module 430, a selection module 440, and a second recognition module 450; wherein,

[0129] The downsizing module 410 is used to downsize the first image in the image frame sequence to obtain the second image;

[0130] The first recognition module 420 is used to perform position recognition on the indicator object in the second image to obtain the indicator position of the indicator object in the second image;

[0131] The update module 430 is used to update the current selected position based on the indicated position to obtain the updated selected position; wherein, the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image;

[0132] The selection module 440 is used to select a third image corresponding to the target position from the first image. The target position includes one of the following: the current selection position and the updated selection position.

[0133] The second recognition module 450 is used to recognize the target object in the third image and obtain the recognition result.

[0134] In some embodiments, the image processing apparatus further includes: a determining module, configured to determine the movement state of the indicated object based on the indicated position and the historical indicated position corresponding to at least one frame image preceding the first image in the image frame sequence; and an output module, configured to: display a second image when the movement state indicates that the indicated object has stopped moving, and / or output the recognition result according to a set output method; the output method includes at least one of the following: display output, voice output.

[0135] In some embodiments, the updating module is further configured to: determine the indicated position as the updated selected position if the indicated position is different from the current selected position.

[0136] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided by the present invention can be used to perform some or all of the steps in the method described in the above method embodiments. For technical details not disclosed in the apparatus embodiments of the present invention, please refer to the descriptions of the method embodiments of the present invention for understanding.

[0137] Exemplary application scenarios

[0138] The following example illustrates a scenario where users can use a terminal to look up words with their finger. In this scenario, the user places the reading material within the camera's field of view and points to the word to be recognized on the material with their finger. The terminal can then recognize the word based on the image captured by the camera and display the recognition result and definition on the screen, thus assisting the user in reading or vocabulary learning. However, some terminals using this technology suffer from slow word lookup speeds. After the user points to a word on the material, there is a noticeable wait before the recognition result appears on the screen. Tests have shown that some terminals using this technology require 1-2 seconds between the user's finger pointing to the word and the display showing the result.

[0139] The inventors analyzed existing solutions for fingertip word lookup in related technologies and found that the app running on the terminal acquires image streams from the camera via the standard Android camera interface, then calls an algorithm module to recognize the finger position and the words in the area pointed to by the finger, and outputs the recognition results to complete the fingertip word lookup. In these solutions, all operations on the image stream are performed on the app's CPU. The image stream needs to be transmitted to the app's Java environment via the Binder interface (an inter-process communication technology), and then the algorithm module is called within the app's process space. This process requires significant system resources and is time-consuming. For example, with a high-resolution camera used for fingertip word lookup at 8MP and 30fps (Frames Per Second), the generated data stream reaches 2.68Gb / s. This data stream, copied via the Binder interface and transmitted to the app layer, consumes a large amount of memory bandwidth, slowing down the entire system's performance.

[0140] To address the problems existing in the aforementioned fingertip word lookup solutions for terminals, the inventors divided the algorithm module into two parts: a finger detection part and an OCR part. The finger detection part can use a scaled-down image as input; for example, an 816*816 pixel (500,000 pixels) image can be used to identify the finger's position. The OCR part can use a 600*600 pixel (360,000 pixels) image cropped based on the finger's pointing position for text recognition. In this way, by scaling down and cropping the first image captured by the camera to obtain a second and third image, and using the finger detection part to detect the finger's position in the second image and the OCR part to recognize the text in the third image, the computational load on the algorithm module can be significantly reduced, and the CPU's processing performance can be improved. Figure 5AThis illustration shows a schematic diagram of an embodiment of the present invention that performs reduction and cropping processing on images captured by a camera, as shown in the figure. Figure 5A As shown, the first image 510 captured by the camera is reduced in size to obtain the second image 520. The second image 520 can be used as input to the finger detection part for finger position recognition. The first image 510 is cropped according to the finger pointing position to obtain the third image 530. The third image 530 can be used as input to the OCR part for text recognition.

[0141] Based on this, at least one of the following methods can be used to further reduce the resource consumption and time required for fingertip word lookup, thereby improving the efficiency of fingertip word lookup:

[0142] 1) Use ISP and RGA respectively for image downsizing and cropping;

[0143] 2) Run the algorithm module on the NPU;

[0144] 3) The driver layer in the standard Android camera framework reconfigures the ISP topology, connecting the common dual-camera path that connects two cameras to the output of one camera (corresponding to the data output channel in the aforementioned embodiment), so that the data from one camera can be output to two video device nodes at the same time, thereby simultaneously acquiring the second image obtained after reduction processing and the third image obtained after cropping processing.

[0145] 4) The second and third images obtained after image downsizing and cropping using ISP and RGA respectively are transmitted to the algorithm module in a zero-copy manner, which greatly reduces memory throughput.

[0146] Figure 5B A schematic flowchart illustrating the implementation of an image processing method according to an embodiment of the present invention is shown, such as... Figure 5BAs shown, the terminal transmits the first image captured by the camera module 512 through the progressive scanning sensor 511 to the image processing controller 513; the image processing controller 513 transmits the first image to the first video device node 514 (corresponding to the image reduction component in the aforementioned embodiment) and the second video device node 515 (corresponding to the image cropping component in the aforementioned embodiment) through the same data output channel; the first video device node 514 transmits the second image obtained after reducing the size of the first image to the finger detection part 516 in the algorithm module; the finger detection part 516 performs position recognition on the finger in the second image to obtain the indication position of the finger in the second image, and transmits the indication position to the second video device node 515; the second video device node 515, based on the currently obtained indication position, crops out a third image corresponding to the indication position from the first image, and transmits the third image to the OCR part 516 in the algorithm module; the OCR part 517 recognizes the text in the third image to obtain the recognition result.

[0147] Figure 5C The diagram schematically illustrates the implementation architecture of an image processing method according to an embodiment of the present invention, such as... Figure 5C As shown, the camera driver 531 in the Linux kernel layer 521 can transmit the first image captured by the camera module through the same data output channel via the image processing controller 532 in the hardware abstraction layer 522 to the first video device node and the second video device node in the image processing module 533 in the library layer 523. The first video device node and the second video device node respectively perform reduction processing and cropping processing on the first image to obtain the second image and the third image. The finger detection part 534 running in the NPU performs position detection on the finger in the second image to obtain the indication position of the finger in the second image, and transmits the indication position to the second video device node for the second video device node to perform the next image cropping processing. Finally, the second image is transmitted to the APP in the application layer 525. 536 is used to provide image preview to the user; the OCR part 535 running in the NPU recognizes the text in the third image, obtains the recognition result, and transmits the recognition result to APP 536 in the application layer 525 for output; the second image and the recognition result are transmitted to the application layer 525 by bypassing the application framework layer 524 in a zero-copy manner, and the second image and the third image are also transmitted to the finger detection part 534 and the OCR part 535 respectively in a zero-copy manner.

[0148] Figure 5D A data flow diagram of an image processing method according to an embodiment of the present invention is illustrated schematically, such as... Figure 5DAs shown, the camera module 541 transmits the raw data stream of the acquired first image to the CSI 542. The CSI 542 converts the raw data stream into a digital signal data form of the first image and outputs the first image to the image splitter 543. The image splitter 543 transmits the received first image to the image processing controller 544. The image controller 544 transmits the first image to the first video device node 545 and the second video device node 546 through the same data output path, and transmits the currently acquired indication position to the second video device node 546. The first video device node 545 uses memory space 551 to perform zero-copy processing on the first image to obtain the second image, which is then transmitted to the finger detection part 547 in the algorithm module. The finger detection part 547 performs position recognition on the finger in the second image to obtain the indication position of the finger in the second image, and transmits the indication position to the image controller 544. The second image is then transmitted to the APP. 549; The second video device node 546, based on the currently acquired indicated position, crops out the third image corresponding to the indicated position from the first image, and uses memory space 552 to transmit the third image to the OCR part 548 in the algorithm module in a zero-copy manner; the OCR part 548 recognizes the text in the third image, obtains the recognition result, and transmits the recognition result to APP 549.

[0149] The image processing method provided in the above embodiments of the present invention can improve the utilization rate of hardware computing power in the terminal, thereby reducing the operating pressure on the processor and improving the overall efficiency of fingertip word lookup.

[0150] Table 1 exemplarily illustrates test data on the processing time of the image processing method provided in the embodiments of the present invention when applied to a fingertip word lookup scenario:

[0151] Table 1. Test data on processing time for fingertip word lookup.

[0152]

[0153]

[0154] As shown in Table 1, when the image processing method provided in this embodiment of the invention is applied to the fingertip word lookup scenario, the overall average time from the finger touching the paper to the output of the word lookup result is 408ms, which greatly improves the overall efficiency of word lookup compared to the 1.5s required for fingertip word lookup in related technologies.

[0155] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above method.

[0156] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0157] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the electronic device and storage medium are similar to those of the above method embodiments, and have similar beneficial effects. For any technical details not disclosed in the embodiments of the electronic device and storage medium of this invention, please refer to the descriptions of the method embodiments of this invention for understanding.

[0158] It should be noted that, Figure 6 The diagram illustrates, schematically, a hardware entity diagram of an electronic device according to an embodiment of the present invention, such as... Figure 6 As shown, the hardware entity of the electronic device 600 includes: a processor 601, a communication interface 602, and a memory 603, wherein:

[0159] The processor 601 typically controls the overall operation of the electronic device 600.

[0160] Communication interface 602 enables electronic devices to communicate with other terminals or servers via a network.

[0161] The memory 603 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 601 and various modules in the electronic device 600. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 601, the communication interface 602, and the memory 603 can be performed via bus 604.

[0162] It should be noted that although several devices or sub-devices of terminals, image processing apparatuses, and electronic devices have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more devices described above can be embodied in one device. Conversely, the features and functions of one device described above can be further divided and embodied by multiple devices.

[0163] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0164] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0165] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0166] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0167] The use of the verbs "including" and "contains" and their inflections in the application documents does not preclude the existence of elements or steps other than those described in the application documents. The article "a" or "one" preceding an element does not preclude the existence of multiple such elements.

[0168] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. An image processing method, characterized in that, Applied to a terminal, the method includes: The first image in the image frame sequence is scaled down to obtain the second image; The location of the indicated object in the second image is identified to obtain the indicated position of the indicated object in the second image; The current selected position is updated based on the indicated position to obtain the updated selected position; wherein the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image; A third image corresponding to the target location is selected from the first image, wherein the target location includes one of the following: the current selected location, the updated selected location; wherein, if the current selected location has not been updated, the current selected location is used as the target location; if the current selected location has been updated, the updated selected location is used as the target location. The target object in the third image is identified, and the identification result is obtained.

2. The method according to claim 1, characterized in that, The method further includes: Based on the indicated position and the historical indicated position corresponding to at least one frame image before the first image in the image frame sequence, the movement state of the indicated object is determined. When the movement state indicates that the indicated object has stopped moving, the second image is displayed, and / or the recognition result is output according to a set output method; the output method includes at least one of the following: display output, voice output.

3. The method according to claim 1, characterized in that, The step of updating the current selected position based on the indicated position to obtain the updated selected position includes: If the indicated position is different from the current selected position, the indicated position will be determined as the updated selected position.

4. The method according to claim 1, characterized in that, The terminal includes a processor and peripheral devices communicatively connected to the processor, the peripheral devices including an image processing module; The step of reducing the size of the first image in the image frame sequence to obtain the second image includes: the image processing module reducing the size of the first image in the image frame sequence to obtain the second image; The step of performing position recognition on the indicator object in the second image to obtain the indicator position of the indicator object in the second image includes: the processor performing position recognition on the indicator object in the second image to obtain the indicator position of the indicator object in the second image; The step of updating the current selected position based on the indicated position to obtain the updated selected position includes: the processor updating the current selected position based on the indicated position to obtain the updated selected position; The step of selecting a third image corresponding to the target position from the first image includes: the image processing module selecting a third image corresponding to the target position from the first image; The step of identifying the target object in the third image and obtaining the identification result includes: the processor identifying the target object in the third image and obtaining the identification result.

5. The method according to claim 4, characterized in that, The method further includes at least one of the following: The image processing module uses a zero-copy method to transmit the second image to the processor; The image processing module uses a zero-copy method to transmit the third image to the processor.

6. The method according to claim 4 or 5, characterized in that, The image processing module includes an image reduction component and an image cropping component; The image processing module reduces the size of a first image in an image frame sequence to obtain a second image, including: the image reduction component reduces the size of a first image in an image frame sequence to obtain a second image; The image processing module selects a third image corresponding to the target position from the first image, including: the image cropping component cropping the third image corresponding to the target position from the first image.

7. The method according to claim 6, characterized in that, The peripheral devices further include an image acquisition module and an image transmission module; the method further includes: The image acquisition module acquires the first image; The image transmission module transmits the first image to the image reduction component and the image cropping component respectively through the same data output channel.

8. A terminal, characterized in that, include: The processor and peripheral devices communicatively connected to the processor, the peripheral devices including an image processing module; wherein... The image processing module is used to: reduce the size of a first image in an image frame sequence to obtain a second image; and select a third image corresponding to the target position from the first image. The processor is configured to: perform position recognition on an indicator object in the second image to obtain the indicator position of the indicator object in the second image; update the current selected position based on the indicator position to obtain the updated selected position; and identify a target object in the third image to obtain an identification result. The target location includes one of the following: the current selected location, the updated selected location; if the current selected location is not updated, the current selected location is used as the target location; if the current selected location has been updated, the updated selected location is used as the target location.

9. The terminal according to claim 8, characterized in that, The image processing module is also used for at least one of the following: The second image is transmitted to the processor using a zero-copy method; The third image is transmitted to the processor using a zero-copy method.

10. The terminal according to claim 8, characterized in that, The processor is also used for: Based on the indicated position and the historical indicated position corresponding to at least one frame image before the first image in the image frame sequence, the movement state of the indicated object is determined. When the movement state indicates that the indicated object has stopped moving, the second image is displayed, and / or the recognition result is output according to a set output method; the output method includes at least one of the following: display output, voice output.

11. The terminal according to any one of claims 8 to 10, characterized in that, The image processing module includes an image reduction component and an image cropping component; wherein... The image downsizing component is used to downsize the first image in the image frame sequence to obtain the second image; The image cropping component is used to crop a third image corresponding to the target position from the first image.

12. The terminal according to claim 11, characterized in that, The peripheral devices also include an image acquisition module and an image transmission module; wherein, The image acquisition module is used to acquire the first image; The image transmission module is used to transmit the first image to the image reduction component and the image cropping component respectively through the same data output channel.

13. The terminal according to any one of claims 8 to 10, characterized in that, The processor includes a central processing unit and a neural network processor; wherein... The neural network processor is configured to: perform position recognition on the indicated object in the second image to obtain the indicated position of the indicated object in the second image; update the current selected position based on the indicated position to obtain the updated selected position; and recognize the target object in the third image to obtain the recognition result, and transmit the recognition result to the central processing unit. The central processing unit is used to output the recognition result according to a set output method; the output method includes at least one of the following: display output, voice output.

14. An image processing apparatus, characterized in that, include: The downsizing module is used to downsize the first image in the image frame sequence to obtain the second image; The first recognition module is used to perform position recognition on the indicator object in the second image to obtain the indication position of the indicator object in the second image; An update module is used to update the current selected position based on the indicated position to obtain the updated selected position; wherein the current selected position is determined based on the indicated position corresponding to the previous frame image of the first image; The selection module is used to select a third image corresponding to a target position from the first image. The target position includes one of the following: the current selection position, the updated selection position; if the current selection position has not been updated, the current selection position is used as the target position; if the current selection position has been updated, the updated selection position is used as the target position. The second recognition module is used to recognize the target object in the third image and obtain the recognition result.

15. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 3.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 3.

Images