Image Processing Method, Apparatus, Device, Storage Medium and Product

Abstract

The present application discloses an image processing method, apparatus, device, storage medium and product, belonging to the field of artificial intelligence technology. The method includes: after obtaining each frame of the image to be processed, detecting whether the image to be processed includes a first object; when it is detected that the first image to be processed includes the first object, displaying a display image corresponding to the first object and operation prompt information corresponding to the display image; when the placement state of the first object meets a preset condition, determining the display image as the target image corresponding to the first object. The embodiments of the present application can be applied to scenarios such as cloud technology, artificial intelligence, and intelligent education. By displaying a partial image corresponding to the first object in the image to be processed, irrelevant display content is reduced; by displaying operation prompt information to prompt the user to adjust the placement state of the first object, and using the display image as the target image when the placement state meets the conditions, the operation steps are simplified and the overall efficiency of image processing is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly relates to an image processing method, apparatus, device, storage medium, and product. Background Art

[0002] With the continuous development of computer technology, intelligent education terminals have been widely applied to daily educational learning life, realizing the integration of electronic products and the education industry, and providing convenience for people's life and learning.

[0003] In related technologies, a student user can place an exercise book under the terminal camera and adjust the position of the exercise book by himself. After the student user adjusts the position of the exercise book, he manually clicks the photo-taking button, and the terminal takes a photo of the exercise within a fixed range and displays the taken exercise photo on the screen for preview before uploading the exercise photo. After seeing the preview photo, the student user judges by himself whether it can be uploaded. If it is judged that it can be uploaded, the student user can manually click the upload button to upload the exercise photo.

[0004] In related technologies, the operation of image photo-taking and uploading is cumbersome, the image quality depends on manual adjustment, the image quality obtained by photo-taking is low, and the overall efficiency is low. Summary of the Invention

[0005] Embodiments of this application provide an image processing method, apparatus, device, storage medium, and product, which can reduce the complexity of image processing operations, improve the overall efficiency of image processing, and optimize the user experience.

[0006] According to one aspect of the embodiments of this application, an image processing method is provided. The method includes:

[0007] After each frame of image to be processed is acquired, detecting whether the image to be processed includes a first object;

[0008] When it is detected that the first image to be processed includes the first object, displaying a display image corresponding to the first object and operation prompt information corresponding to the display image; wherein, the display image is an image corresponding to a positioning area where the first object is located in the first image to be processed, and the operation prompt information is used to prompt a target object to adjust the placement state of the first object;

[0009] When the placement state of the first object meets a preset condition, determining the display image as the target image corresponding to the first object.

[0010] According to one aspect of the embodiments of this application, an image processing apparatus is provided. The apparatus includes:

[0011] An object detection module, configured to detect whether the image to be processed includes a first object after each frame of the image to be processed is obtained;

[0012] An information display module, configured to display a display image corresponding to the first object and operation prompt information corresponding to the display image when it is detected that the first image to be processed includes the first object; wherein, the display image is an image corresponding to a positioning area where the first object is located in the first image to be processed, and the operation prompt information is used to prompt the target object to adjust the placement state of the first object;

[0013] An image determination module, configured to determine the display image as the target image corresponding to the first object when the placement state of the first object meets a preset condition.

[0014] According to one aspect of the embodiments of the present application, a computer device is provided. The computer device includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above image processing method.

[0015] According to one aspect of the embodiments of the present application, a computer-readable storage medium is provided. At least one instruction, at least one program, a code set or an instruction set is stored in the storage medium. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above image processing method.

[0016] According to one aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the above image processing method.

[0017] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0018] By detecting whether the acquired image to be processed contains a first object after acquiring a frame of the image to be processed, it is possible to quickly detect the image to be processed that contains the first object; when it is detected that the first image to be processed contains the first object, a partial image corresponding to the positioning area where the first object is located in the image to be processed will be displayed, effectively reducing the image content unrelated to the first object in the image to be processed and improving the image quality; and corresponding operation prompt information will also be displayed to prompt the user to adjust the placement state of the first object in a timely manner. When the placement state meets the preset conditions, the displayed image can be determined as the target image, reducing the complexity of the image processing operation, improving the overall efficiency of the image processing, and optimizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 Exemplarily shows a page interaction flowchart of a homework photo marking function;

[0021] Figure 2 Is a schematic diagram of an application program running environment provided by an embodiment of the present application;

[0022] Figure 3 Exemplarily shows a schematic diagram of an intelligent education desk lamp;

[0023] Figure 4 Is the flowchart of an image processing method provided by an embodiment of the present application Figure 1 ;

[0024] Figure 5 Is the flowchart of an image processing method provided by an embodiment of the present application Figure 2 ;

[0025] Figure 6 Shows a schematic diagram of a scenario for object display by tracking the object position provided by an embodiment of the present application;

[0026] Figure 7 Exemplarily shows a schematic diagram of a homework shooting scenario;

[0027] Figure 8 Exemplarily shows a schematic diagram of previewing a homework image on a homework photo-taking page;

[0028] Figure 9 Exemplarily shows a schematic diagram of displaying operation prompt information according to the object placement state;

[0029] Figure 10 is the flow of an image processing method provided by an embodiment of the present application Figure 3 ;

[0030] Figure 11 Exemplarily shows a flowchart of job photo marking and correction;

[0031] Figure 12 is the flow of an image processing method provided by an embodiment of the present application Figure 4 ;

[0032] Figure 13 is the block diagram of an image processing apparatus provided by an embodiment of the present application;

[0033] Figure 14 is the structural block diagram of a computer device provided by an embodiment of the present application;

[0034] Figure 15 is the structural block diagram of a computer device provided by another embodiment of the present application. Detailed implementation manners

[0035] The image processing method provided by the embodiments of the present application relates to artificial intelligence technology, which is briefly described below for the understanding of those skilled in the art.

[0036] Artificial Intelligence (AI) is to use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0037] Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, autonomous driving, and intelligent transportation.

[0038] Computer Vision Technology (CV) Computer vision is a science that studies how to enable machines to "see". More specifically, it refers to machine vision that uses cameras and computers to replace human eyes for object recognition, measurement, etc., and further performs graphic processing to make the computer-processed images more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, etc. technologies, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition. In the embodiments of the present application, object recognition, text recognition, image retrieval, etc. can be performed on the collected images based on the above computer vision technology to obtain relevant processing results.

[0039] The key technologies of Speech Technology include automatic speech recognition technology, speech synthesis technology, and voiceprint recognition technology. Enabling computers to listen, see, speak, and feel is the future development direction of human-computer interaction. Among them, speech has become one of the most promising human-computer interaction methods in the future. In the embodiments of the present application, the terminal can recognize the collected audio. For example, according to the user's audio, the operation instructions therein can be recognized. For example, when the user says "take a picture", after the terminal recognizes "take a picture" according to the above speech technology, it takes a picture of the target object to obtain a target image.

[0040] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers in natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language people use in daily life, so it has a close connection with the research of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph, etc. technologies. In the embodiments of the present application, the terminal can perform text recognition on the collected images, and can further perform natural language processing on the recognized text to obtain processing results. For example, in the intelligent education scenario, after the terminal captures a homework problem, it can automatically output the corresponding standard answer to the problem, or correct the recognized answer according to the standard answer.

[0041] Machine Learning (ML) is an interdisciplinary field that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning. In the embodiments of this application, relevant information processing tasks can be performed by training a machine learning model, and the embodiments of this application do not limit this.

[0042] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, autonomous driving, and intelligent transportation. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role. Smart education is a typical application scenario in the embodiments of this application. In the smart education scenario, the above artificial intelligence technology can play an important role in education tutoring tasks. For example, intelligent homework grading, intelligent problem-solving, and intelligent question searching can all be achieved by processing the collected images.

[0043] In addition, the image processing method provided in the embodiments of this application also involves the field of cloud technology, such as the following cloud education scenario.

[0044] Cloud Computing Education (abbreviated as CCEDU) refers to an education platform service based on the cloud computing business model. On the cloud platform, all educational institutions, training institutions, enrollment service institutions, publicity institutions, industry associations, management institutions, industry media, legal structures, etc. are centrally integrated into a resource pool. Each resource can display and interact with each other, communicate as needed, and reach an agreement, thereby reducing education costs and improving efficiency.

[0045] The following briefly introduces the relevant terms or nouns that may be involved in the embodiments of this application to facilitate the understanding of those skilled in the art of this application.

[0046] A smart education desk lamp is an intelligent screen-equipped desk lamp product that assists students in learning. Generally, it is equipped with two cameras. The camera facing downwards at the lamp head is used to photograph the desk and homework, and the camera facing forward is used to photograph people's sitting postures and for video calls. It usually supports functions such as taking pictures and grading homework, detecting bad sitting postures, and fingertip reading of words.

[0047] Homework photographing refers to using the top camera of the desk lamp to photograph the homework on the desk, uploading it to the background for homework grading, or submitting it to the client on the parent's or teacher's side.

[0048] Homework grading refers to performing OCR (Optical Character Recognition) on the homework pictures uploaded by students through photographing, converting the picture content into text, identifying the subjects, question types, etc., and automatically grading the students' answers. Grading has requirements for the resolution, clarity of the pictures, the position of the homework books, etc. The better the quality of the pictures, the higher the accuracy of the grading results.

[0049] In one example, as Figure 1 shown, it exemplarily shows a page interaction flow chart of a homework photographing and grading function. The student user can place the homework book under the terminal camera and adjust the position of the homework book by himself / herself. The captured image of the camera can be displayed in the preview area 02 of page 01. After the student user adjusts the position of the homework book, manually click the photographing button 03 on page 01, and the terminal will photograph the homework within a fixed range and display the photographed homework picture in the preview area 02 for preview before uploading the homework picture. After seeing the preview picture, the student user can judge by himself / herself whether it can be uploaded. If it is judged that it can be uploaded, the student user can manually click the upload and grading button 04 on page 01 to upload the homework picture.

[0050] Photographing viewfinder area: During the process of photographing homework, the actual preview area of the camera will be displayed on the screen of the desk lamp in real time, and the user can clearly see the effect after photographing, which is generally a fixed rectangular area.

[0051] Image resolution: The amount of information stored in the image, that is, how many pixel points are there in each inch of the image. The higher the resolution, the clearer the picture.

[0052] Edge detection: Edge detection is a method of analyzing images in image processing. The purpose is to find the set of pixel points with drastic brightness changes in the image, which generally shows the outline of the object; if the outline can be accurately detected, the shape of the object can be measured.

[0053] Motion detection: (Motion Detection) refers to detecting the changed area in a sequence of images and extracting the moving object from the background image, which is usually used to detect the moving / static state of a specific object.

[0054] To make the purpose, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0055] Please refer to Figure 2, which shows a schematic diagram of an application program running environment provided by an embodiment of the present application. The application program running environment may include: a terminal 10 and a server 20.

[0056] The terminal 10 includes but is not limited to electronic devices such as mobile phones, computers, intelligent voice interaction devices, intelligent speakers, smart watches, smart TVs, smart table lamps, smart home appliances, vehicle-mounted terminals, game consoles, e-book readers, multimedia playback devices, wearable devices, etc. A client of the application program can be installed in the terminal 10.

[0057] In an embodiment of the present application, the above application program can be any application program that can provide intelligent education services. Typically, the application program is a learning application program. Of course, in addition to learning application programs, intelligent education services can also be provided in other types of application programs. For example, education management application programs, homework correction application programs, teaching assistant application programs, interactive education application programs, office application programs, virtual reality (VR) application programs, augmented reality (AR) application programs, etc. The embodiments of the present application do not limit this.

[0058] In an exemplary embodiment, the above terminal 10 may be an intelligent education table lamp. Please refer to Figure 3 , Figure 3 An exemplary schematic diagram of an intelligent education table lamp is shown. In a possible implementation, the intelligent education table lamp 101 is equipped with a camera 1011, a light sensor 1012, and a display 1013. The camera 1011 is used to collect images, and the collected images will be transmitted to an image processor mounted inside the intelligent education table lamp 101 for processing (the image processor can also be mounted inside the display 1013). The light sensor 1012 is used to detect the ambient light intensity and to perform supplementary light assistance processing when the supplementary light condition is met. The display 1013 is used to display a user interface, and the display 1013 can be a touch display screen, which can not only display the user interface but also receive user operations. As Figure 2As shown in the figure, the camera 1011 is installed on the rod of the intelligent education desk lamp 101. Practice has found that installing the camera 1011 on the rod of the intelligent education desk lamp 101 can not only collect the homework image more completely, but also ensure higher clarity of the collected image. The light sensor 1012 is located beside the camera 1011. Specifically, it can be at any position around the camera 1011. Installing the light sensor 1012 beside the camera 1011 helps to more accurately detect the light intensity of the surrounding environment when the camera collects images, facilitating light compensation when the light compensation condition is met. The display 1013 is located above the base of the intelligent education desk lamp 101. In one implementation, the display 1013 is also equipped with a sound output device (such as a speaker) to play operation prompt information to the user during the process of placing the homework, so as to optimize the placement state of the homework and then improve the quality of the photographed image. In addition, the intelligent education desk lamp 101 is also equipped with a light source 1014. It can be understood that the intelligent education desk lamp 101 can perform light compensation (enhance the light intensity of the surrounding environment) through the light source 1014; when the intelligent function is not used, the intelligent education desk lamp 101 can be used as an ordinary desk lamp. The device main body 1015 of the intelligent education desk lamp 101 can also be equipped with a sound output device, a storage battery, and a data transmission interface (such as a USB interface, a charging interface, etc.), and this application does not limit this. In a typical application scenario, such as the scenario of taking a photo of homework and uploading it, the intelligent education desk lamp 101 can collect the homework image through the camera 1011. After collecting the homework image, it can identify the position of the homework in the homework image or the position of the user's finger in the homework image. The homework can be located through the above two positions, so as to display the homework content at the maximum resolution on the display 1013.

[0059] In Figure 2 the example, the server 20 is used to provide background services for the client of the application program in the terminal 10. For example, the server 20 can be the background server of the above application program. The server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the server 20 provides background services for the application programs in multiple terminals 10 at the same time.

[0060] Optionally, the terminal 10 and the server 20 can communicate with each other through the network 30. The terminal 10 and the server 20 can be directly or indirectly connected through wired or wireless communication methods, and this application does not limit this.

[0061] Please refer to Figure 4 , which shows the flowchart of the image processing method provided by an embodiment of the present application Figure 1 . This method can be applied to a computer device, which refers to an electronic device with data calculation and processing capabilities. For example, the execution entity of each step can be Figure 2 the terminal 10 in the application program running environment shown. This method may include the following steps (410-430).

[0062] Step 410, after obtaining each frame of the image to be processed, detect whether the image to be processed includes a first object.

[0063] Optionally, the image to be processed is an image extracted from a sequence of image frames collected by a camera at a preset frequency. Correspondingly, after extracting each frame of image from the sequence of image frames, use it as the above-mentioned image to be processed and detect whether the image to be processed includes a first object.

[0064] Optionally, the above-mentioned first object is a shooting object. The present application embodiment does not limit the type, shape, etc. of the first object, and the first object can be any shooting object. In the intelligent education scenario, the above-mentioned first object includes but is not limited to shooting objects such as student homework, test questions, books, etc.

[0065] In an exemplary embodiment, as Figure 5 shown, the implementation process of the above step 410 includes the following steps (411-412), Figure 5 which shows the flowchart of the image processing method provided by an embodiment of the present application Figure 2 .

[0066] Step 411, after obtaining each frame of the image to be processed, perform first object detection processing on the image to be processed to obtain a first detection result.

[0067] In a possible implementation manner, the above-mentioned first object detection processing is edge detection processing corresponding to the first object. After obtaining each frame of the image to be processed, perform edge detection processing on the image to be processed to obtain a first detection result. The above-mentioned first detection result includes edge information corresponding to the first object. Optionally, the above-mentioned edge information includes the coordinates of the positioning points corresponding to the first object. Optionally, the above-mentioned positioning point is the vertex corresponding to the first object.

[0068] If the image to be processed includes a complete first object, then the first detection result includes the overall edge information of the first object. The above-mentioned overall edge information includes the coordinate information of all positioning points corresponding to the first object. For example, if the shape of the first object is a rectangle, the coordinate information of all positioning points corresponding to the first object includes the coordinate information of the four vertices corresponding to the rectangle.

[0069] If the to-be-processed image includes a partial first object, then the first detection result includes the local edge information of the first object. The above local edge information includes the coordinate information of some positioning points corresponding to the first object. Here, taking the shape of the first object as a rectangle as an example, the coordinate information of some positioning points of the first object includes the coordinate information of some vertices among the four vertices corresponding to the rectangle.

[0070] Step 412, if the first detection result corresponding to the first to-be-processed image indicates that the first to-be-processed image includes the overall edge information or local edge information of the first object, it is determined that the first to-be-processed image includes the first object.

[0071] In a possible implementation manner, the situation where the to-be-processed image includes the first object can be divided into two types. One is that the to-be-processed image includes the complete first object, that is, the to-be-processed image includes the overall edge information of the first object; the other is that the to-be-processed image includes an incomplete first object, that is, the to-be-processed image includes the local edge information of the first object.

[0072] Step 420, when it is detected that the first to-be-processed image includes the first object, display the display image corresponding to the first object and the operation prompt information corresponding to the display image.

[0073] Among them, the display image is the image corresponding to the positioning area where the first object is located in the first to-be-processed image, and the operation prompt information is used to prompt the target object to adjust the placement state of the first object.

[0074] Optionally, the above positioning area is a partial area of the to-be-processed image that includes the first object. The above partial area can be the smallest area of the to-be-processed image that includes the first object. The above positioning area includes all the edge information corresponding to the first object in the to-be-processed image, and the image content outside the positioning area in the to-be-processed image is the content image unrelated to the first object. For example, if the above first object is a rectangle, then the first object may occupy a rectangular area in the to-be-processed image, and the terminal only needs to display the image content corresponding to the rectangular area. The image area occupied by the first object in the above display image is greater than or equal to the preset ratio threshold. The first object is in the center position in the above display image.

[0075] In a possible implementation manner, displaying the display image corresponding to the first object includes: displaying the above display image in the preview area of the preset page.

[0076] The above operation prompt information includes prompt information of at least one media type, and the above media type includes but is not limited to text type, image type, audio type, and video type.

[0077] In a possible implementation, presenting operation prompt information includes: when the operation prompt information is of text type, presenting text prompt information on the above-mentioned preset page; when the operation prompt information is of image or video type, presenting image prompt information or video prompt information on the above-mentioned preset page; when the operation prompt information is of audio type, playing voice prompt information.

[0078] Optionally, the above-mentioned operation prompt information includes angle adjustment prompt information, corner curling adjustment prompt information, position adjustment prompt information, and stationary prompt information. The above-mentioned angle adjustment prompt information is used to prompt the target object to adjust the placement angle of the first object; the above-mentioned corner curling adjustment prompt information is used to prompt the target object to adjust the corner curling of the first object; the above-mentioned position adjustment prompt information is used to prompt the target object to adjust the placement position of the first object; the above-mentioned stationary prompt information is used to prompt the target object to place the first object stationary. Optionally, each type of operation prompt information corresponds to prompt information of various media types.

[0079] By only presenting the image corresponding to the positioning area where the first object is located in the image to be processed, tracking display of the first object can be achieved. No matter where the first object is in the image to be processed, it can be ensured that the first object is centered and presented at the maximum resolution in the page preview area, and the irrelevant image content outside the positioning area is cancelled from display, which can effectively improve the quality of the presented image.

[0080] In an exemplary embodiment, as Figure 5 shown, the implementation process of the above-mentioned step 420 includes the following steps (421 to 423).

[0081] Step 421, if the first image to be processed includes the overall edge information of the first object, then determine the positioning area and the placement angle of the first object based on the overall edge information.

[0082] In a possible implementation, determine the above-mentioned positioning area based on the coordinate information of the positioning points in the overall edge information. Optionally, determine the above-mentioned positioning area based on the coordinate information of the vertices in the overall edge information. Optionally, based on the overall edge information of the first object, determine the maximum longitudinal spacing corresponding to the first object in the longitudinal direction and the maximum transverse spacing corresponding to the first object in the transverse direction. Based on the above-mentioned maximum longitudinal spacing and maximum transverse spacing, the positioning area corresponding to the first object can be determined. For example, extend the boundary determined by the maximum longitudinal spacing and maximum transverse spacing outward by a preset pixel value to obtain the above-mentioned positioning area. Optionally, the above-mentioned preset pixel value is 20 pixels.

[0083] In a possible implementation manner, based on the coordinate information of the positioning points in the overall edge information, the above-mentioned placement angle is determined. Optionally, based on the coordinate information of the vertices in the overall edge information, the above-mentioned placement angle is determined. For example, based on the coordinates of the vertices, the connection lines between the vertices can be determined, that is, the object edges; the angle formed by the connection line between the vertices and the horizontal direction, or the angle formed by the connection line between the vertices and the vertical direction is determined as the above-mentioned placement angle.

[0084] Step 422: Display the display image corresponding to the positioning area.

[0085] Optionally, the above-mentioned display image is displayed in a preset area of a preset page.

[0086] In a possible application scenario, the above-mentioned first object is placed obliquely in the image to be processed. Therefore, an oblique positioning area can be determined, and then when it is displayed in the preset area, the oblique positioning area is rotated, and the display image corresponding to the rotated oblique positioning area is displayed in the above-mentioned preset area.

[0087] In an example, as Figure 6 shown, Figure 6 FIG. shows a schematic diagram of a scenario for object display by tracking the object position provided by an embodiment of the present application. In Figure 6 the example, the above-mentioned first object is an exercise book 61, and the device for object display is an intelligent education desk lamp 62. The intelligent education desk lamp 62 can automatically track the position of the exercise book 62 within its shooting range, and determine the positioning area 63 where the exercise book 62 is located according to the position of the exercise book 62, and then can display the image corresponding to the positioning area 63 in the preview area 64 of the screen, so that the exercise book 62 can always be located at the center of the preview area 64 and be displayed in the preview area 64 at the maximum resolution.

[0088] In another example, as Figure 7 shown, Figure 7 FIG. exemplarily shows a schematic diagram of a scenario for homework shooting. In Figure 7 , the preview area 71 of the homework photographing device is strictly restricted, generally located directly below the device and fixed in position. Students must manually place the exercise book 72 directly below the device and then click the shooting button to take a photo, otherwise problems such as incomplete shooting may occur. In this way, the picture quality completely depends on the operation and placement specifications of the students. For example, if the student places the exercise book far from the desk lamp, the resolution of the exercise book will decrease, and there may be problems such as the placement direction of the exercise book, incomplete corners, and curling of the homework, resulting in a decrease in the accuracy of background correction. As Figure 8 shown, Figure 8The figure exemplarily shows a schematic diagram of previewing a homework image on a homework photographing page. In the preview page 81, since the distance between the exercise book 72 and the camera is far, the size of the exercise book 72 in the preview page 81 is small; in the preview page 82, since Figure 7 the position of the preview area 71 is fixed in the preview page, and the student does not place the exercise book 72 in the preview area 71, resulting in a missing corner of the exercise book 72 in the preview page 82; since the student does not place the exercise book 72 in the preview area 71, the problem that the angle of the exercise book 72 is incorrect in the preview page 83; in the preview page 84, there is a problem that the exercise book 72 has a folded corner or is bent.

[0089] Compared with Figure 7 the corresponding operation shooting scheme of the example, Figure 6 the corresponding method of previewing and shooting by tracking the position of the operation in the example can always keep the exercise book at the center of the preview area and display it in the preview area at the maximum resolution when the exercise book is within the shooting range of the camera, effectively improving the image quality.

[0090] Step 423, when the placement angle is greater than the angle threshold, display angle adjustment prompt information.

[0091] The operation prompt information includes angle adjustment prompt information.

[0092] If the above-mentioned placement angle is greater than the angle threshold, it means that the first object is not placed correctly, and then the above-mentioned angle adjustment information can be displayed to prompt the target object to place the first object correctly. Optionally, the above-mentioned angle threshold is 10 degrees.

[0093] In a possible implementation manner, the above-mentioned angle adjustment prompt information includes angle adjustment voice prompt information. Correspondingly, when the placement angle is greater than the angle threshold, play the angle adjustment voice prompt information. For example, the angle adjustment voice prompt information is "The direction is incorrect. Please place the first object correctly." Correspondingly, the above-mentioned angle adjustment prompt information also includes angle adjustment text prompt information. When the placement angle is greater than the angle threshold, display the angle adjustment text prompt information on the preset page.

[0094] In a possible implementation manner, the above-mentioned angle adjustment prompt information further includes border prompt information. Correspondingly, when the placement angle is greater than the angle threshold, add a border of the first style to the first object in the preview area. For example, add a red border to the edge of the first object in the displayed image.

[0095] In an exemplary embodiment, as Figure 5 shown, the implementation process of the above-mentioned step 420 further includes the following steps (424 to 425).

[0096] Step 424: Perform a corner curling detection process on the displayed image to obtain the corner curling detection result of the first object.

[0097] In a possible implementation manner, process the displayed image according to the ellipse detection algorithm to detect whether the displayed image includes a corner curl.

[0098] In a possible implementation manner, input the displayed image into a corner curling detection model for corner curling detection processing to obtain the corner curling detection result of the first object. Optionally, the corner curling detection model is a machine learning model trained based on training sample images and the labels corresponding to the sample images. The above sample images include object images containing corner curls, and the above labels indicate whether the images include corner curls.

[0099] The above corner curling detection result is used to indicate whether the first object has a corner curl. The corner curling detection result includes two cases. One is that the first object has a corner curl; the other is that the first object does not have a corner curl. In the case where the first object has a corner curl, the corner curling detection result further includes corner curl positioning information for indicating the corner curl position.

[0100] Step 425: When the corner curling detection result indicates that the first object has a corner curl, display a corner curl adjustment prompt message.

[0101] The operation prompt message includes a corner curl adjustment prompt message.

[0102] In a possible implementation manner, the above corner curl adjustment prompt message includes a corner curl adjustment voice prompt message. Correspondingly, when the corner curling detection result indicates that the first object has a corner curl, play the corner curl adjustment voice prompt message. For example, the corner curl adjustment voice prompt message is "The exercise book is curled. Please keep the four corners intact." Correspondingly, the above corner curl adjustment prompt message further includes a corner curl adjustment text prompt message. When the corner curling detection result indicates that the first object has a corner curl, display the corner curl adjustment text prompt message on a preset page.

[0103] In a possible implementation manner, the above corner curl adjustment prompt message further includes a border prompt message. Correspondingly, when the corner curling detection result indicates that the first object has a corner curl, add a border of the first style to the first object in the preview area. For example, add a red border to the edge of the first object in the displayed image.

[0104] In an exemplary embodiment, as Figure 5 shown, the implementation process of the above step 420 further includes the following steps (426 to 427).

[0105] Step 426: If the first image to be processed includes local edge information of the first object, determine a positioning area based on the local edge information.

[0106] In a possible implementation, based on the coordinate information of the positioning points in the local edge information, the above-mentioned positioning area is determined. Optionally, based on the coordinate information of the vertices in the local edge information, the above-mentioned positioning area is determined.

[0107] Step 427: Display the display image corresponding to the positioning area and the position adjustment prompt information corresponding to the local edge information.

[0108] The operation prompt information includes the position adjustment prompt information. In the display image corresponding to the positioning area determined based on the local edge information, an incomplete first object is included.

[0109] In a possible implementation, the above-mentioned position adjustment prompt information includes position adjustment voice prompt information. Correspondingly, when the first image to be processed includes the local edge information of the first object, the position adjustment voice prompt information is played. For example, the position adjustment voice prompt information is "The complete exercise book is not detected. Please adjust the position." Correspondingly, the above-mentioned position adjustment prompt information further includes position adjustment text prompt information. When the first image to be processed includes the local edge information of the first object, the position adjustment text prompt information is displayed on the preset page.

[0110] In a possible implementation, the above-mentioned position adjustment prompt information further includes border prompt information. Correspondingly, when the first image to be processed includes the local edge information of the first object, a border of the first style is added to the first object in the preview area. For example, a red border is added to the local edge of the first object in the display image.

[0111] In an example, as Figure 9 shown, it exemplarily shows a schematic diagram of displaying operation prompt information according to the object placement state. Optionally, Figure 9The first object in it is an exercise book. The embodiments of the present application are applied to the scenario of photographing and correcting homework, which can detect the correctness of the placement of the homework in real time. If it is incorrect, corresponding voice prompts will be given, and a red frame will be drawn on the edge of the exercise book; if it is correct, a green frame will be drawn on the edge of the exercise book, and a correct voice prompt will be given. For example, when the exercise book 91 in the preview area 95 is incomplete, a red border is drawn on the edge of the exercise book 91, and the first voice prompt message "A complete exercise book is not detected. Please adjust the position" is played; when the exercise book 92 in the preview area 95 is not placed correctly, a red border is drawn on the edge of the exercise book 92, and the second voice prompt message "The direction is incorrect. Please place the exercise book correctly" is played; when the exercise book 93 in the preview area 95 has a corner curl, a red border is drawn on the edge of the exercise book 93, and the third voice prompt message "The exercise book has a curl. Please keep all four corners intact" is played; when the exercise book 94 in the preview area 95 is placed correctly, a green border is drawn on the edge of the exercise book 94, and the fourth voice prompt message "The placement is correct. Please stay still" is played.

[0112] Step 430, when the placement state of the first object meets the preset conditions, determine the displayed image as the target image corresponding to the first object.

[0113] The above-mentioned preset conditions refer to the conditions for judging the placement state. The above-mentioned placement state is used to characterize the display quality of the object.

[0114] The above-mentioned placement state includes the object edge state, placement angle state, corner curl state, and motion state of the first object.

[0115] The above-mentioned preset conditions include the object integrity condition, placement angle condition, corner curl condition, and stillness condition. Optionally, the above-mentioned object integrity condition is that the image includes complete edge information. Optionally, the above-mentioned placement angle condition is that the placement angle is less than or equal to the angle threshold. Optionally, the corner curl condition is that there is no corner curl. Optionally, the stillness condition is that the object is in a still state.

[0116] In an exemplary embodiment, as Figure 5 shown, the implementation process of the above-mentioned step 430 further includes the following steps (431 to 434).

[0117] Step 431, when the placement angle is less than or equal to the angle threshold and the corner curl detection result indicates that there is no corner curl in the first object, perform motion detection processing on the first object to obtain a motion detection result.

[0118] Optionally, compare the position information of the first object in the current image to be processed with the position information of the first object in the previous image to be processed to obtain a motion detection result.

[0119] If the position of the first object changes between the current image to be processed and the previous image to be processed, it is determined that the first object is in a moving state; if the position of the first object does not change between the current image to be processed and the previous image to be processed, it is determined that the first object is in a stationary state.

[0120] The above-mentioned previous object to be processed refers to the image to be processed before the current image to be processed.

[0121] Step 432, if the motion detection result indicates that the first object is in a stationary state, the displayed image is determined as the target image.

[0122] The first object stops moving, indicating that the target object has completed the placement of the first object, and then the photographing process can be performed to obtain a snapshot image of the first object. Optionally, the above-mentioned target image is a snapshot image of the first object.

[0123] Step 433, if the motion detection result indicates that the first object is in a moving state, a stationary prompt message is displayed.

[0124] In a possible implementation manner, the above-mentioned stationary prompt message includes a stationary voice prompt message. Correspondingly, when the motion detection result indicates that the first object is in a moving state, the stationary voice prompt message is played. For example, the stationary voice prompt message is "The placement is correct, please stay still". Correspondingly, the above-mentioned stationary prompt message further includes a stationary prompt text message, and when the motion detection result indicates that the first object is in a moving state, the stationary prompt text message is displayed on a preset page.

[0125] In a possible implementation manner, the above-mentioned stationary prompt message further includes a border prompt message. Correspondingly, when the motion detection result indicates that the first object is in a moving state, a border of a second style is added to the first object in the preview area. For example, a green border is added to the local edge of the first object in the displayed image.

[0126] Step 434, obtain the next image to be processed. And start executing from the above step 411 again.

[0127] If the first object is in a moving state, it means that the adjustment process of the target object for the first object has not ended yet. Only the above-mentioned operation prompt message is displayed, and no photographing is performed. Just obtain the next frame and repeat the above process; if the first object is in a stationary state for a certain period of time and is placed correctly, the photographing is automatically triggered, the displayed image is determined as the target image, and the target image can be sent to the subsequent process, such as uploading, without any click operation by the user, reducing the operation complexity.

[0128] In an exemplary embodiment, as Figure 5 shown, after the above step 430, the following step (440) is further included.

[0129] Step 440: Feed the target image into the target information recognition process to obtain the information recognition result corresponding to the target image.

[0130] The above target information recognition process is any information recognition process. In the above information recognition process, multimedia processing is performed on the above target image to obtain the information recognition result corresponding to the target image. The embodiments of the present application do not limit the information recognition process and the multimedia processing method, which can be set according to specific application scenarios.

[0131] In the intelligent education scenario, the first object is a student's homework. Correspondingly, the above information recognition process can be a homework grading process.

[0132] In summary, the technical solution provided by the embodiments of the present application can quickly detect a to-be-processed image containing a first object after obtaining a frame of to-be-processed image, and detect whether the obtained to-be-processed image contains the first object; when it is detected that the first to-be-processed image contains the first object, a partial image corresponding to the positioning area where the first object is located in the to-be-processed image will be displayed, effectively reducing the image content irrelevant to the first object in the to-be-processed image and improving the image quality; and corresponding operation prompt information will also be displayed to prompt the user to adjust the placement state of the first object in a timely manner. When the placement state meets the preset conditions, the displayed image can be determined as the target image, reducing the complexity of the image processing operation, improving the overall efficiency of the image processing, and optimizing the user experience.

[0133] Please refer to Figure 10 , which shows the flow of the image processing method provided by an embodiment of the present application Figure 3 . This method can be applied to a computer device, which refers to an electronic device with data calculation and processing capabilities. For example, the execution entity of each step can be Figure 2 the terminal 10 in the application program running environment shown. This method may include the following steps (1001-1016).

[0134] Step 1001: After obtaining each frame of to-be-processed image, detect whether the to-be-processed image includes a first object.

[0135] After obtaining each frame of to-be-processed image, perform first object detection processing on the to-be-processed image to obtain a first detection result.

[0136] If the first detection result corresponding to the second to-be-processed image indicates that the second to-be-processed image includes the overall edge information or partial edge information of the first object, it is determined that the second to-be-processed image includes the first object.

[0137] Step 1002, when it is detected that the second image to be processed includes the local edge information of the first object, determine the edge missing direction corresponding to the first object in the second image to be processed according to the local edge information.

[0138] The above-mentioned edge missing direction refers to the direction corresponding to the partial edge of the first object missing in the image to be processed. For example, if the image to be processed only includes the left half of the first object and lacks the right half of the first object, then the above-mentioned edge missing direction is the left side of the image to be processed.

[0139] In a possible implementation manner, determine the above-mentioned edge missing direction according to the coordinate information of the positioning points in the local edge information. The above-mentioned edge missing direction can be determined according to the coordinates of the positioning points corresponding to the first object in the image to be processed.

[0140] In a possible implementation manner, determine the above-mentioned edge missing direction according to the local object contour information in the local edge information. The overall contour of the first object can usually be regarded as a closed contour, and the above-mentioned local object contour usually cannot form a closed contour. Then, the position of the contour gap can be determined according to the local object contour, and the above-mentioned edge missing direction can be determined based on the position of the contour gap.

[0141] In a possible implementation manner, the above-mentioned edge missing direction can be determined by judging the image edge in contact with the partial edge of the first object in the image to be processed. In the embodiments of the present application, the reason why the image to be processed does not include the complete first object is usually that the viewing area of the camera viewfinder fails to completely cover the area where the first object is located, rather than the reason that the first object is blocked. Therefore, the partial edge of the first object included in the image to be processed is in contact with the image edge, and the first object is also located at the boundary of the camera viewfinder. Therefore, according to the above-mentioned local edge information, the image edge corresponding to the partial edge of the first object in the image to be processed can be determined, and the direction where the image edge is located is determined as the above-mentioned edge missing direction.

[0142] Step 1003, move the image viewfinder in the camera shooting area from the first position to the second position according to the edge missing direction.

[0143] Wherein, the first position is the current position of the image viewfinder, and the second position is the position after the image viewfinder is moved.

[0144] The above-mentioned camera shooting area refers to the maximum shootable area corresponding to the camera, which is the limit of the camera shooting range. The above-mentioned camera viewfinder refers to the area used by the camera to determine the viewing area of the camera in the camera shooting area. The above-mentioned image to be processed is the image in the viewing area and is not necessarily the image of the maximum range that the camera can shoot.

[0145] Optionally, in the camera shooting area, the image viewfinder is moved by a target distance from the first position towards the edge missing direction to reach the second position. The above target distance can be a fixed preset value or a moving distance determined according to the above local edge information. Optionally, the unit of the above target distance is pixels.

[0146] Before performing the above step 1003, it can be first determined whether the image viewfinder has reached the edge of the camera shooting area.

[0147] Step 1004, when the image viewfinder is in the second position, image acquisition is performed on the camera shooting area to re-obtain the image to be processed.

[0148] And then start executing from the above step 1001 again.

[0149] After the image viewfinder is moved to the second position, image acquisition is re-performed on the camera shooting area, and the acquired image frame is added to the image frame sequence corresponding to the camera. The device can continue to extract image frames from the image frame sequence as the image to be processed at the preset frame rate. Only at this time, the image to be processed extracted is the image of the view area corresponding to the image viewfinder in the second position in the camera shooting area.

[0150] Step 1005, when it is detected that the first image to be processed includes the local edge information of the first object and the image viewfinder reaches the area edge of the camera shooting area, the positioning area is determined based on the local edge information.

[0151] If the first object still cannot be captured or a complete first object cannot be captured even when the image viewfinder is moved to the area edge of the camera shooting area, it means that the position of the first object is outside the camera shooting area. In addition to displaying the display image of the partial first object, a position adjustment prompt message needs to be sent to the user so that the user can move the first object into the camera shooting area.

[0152] Step 1006, display the display image corresponding to the positioning area and the position adjustment prompt message corresponding to the local edge information.

[0153] Among them, the first image to be processed is the image acquired when the image viewfinder reaches the area edge, and the operation prompt message includes the position adjustment prompt message.

[0154] Step 1007, when it is detected that the third image to be processed includes the overall edge information of the first object, the positioning area and the placement angle of the first object are determined based on the overall edge information.

[0155] The above first image to be processed, second image to be processed, and third image to be processed are all different image frames in the image frame sequence acquired by the camera.

[0156] Step 1008: Display the display image corresponding to the positioning area.

[0157] Step 1009: When the placement angle is greater than the angle threshold, display the angle adjustment prompt message.

[0158] The operation prompt message includes the angle adjustment prompt message.

[0159] Step 1010: When the placement angle is less than or equal to the angle threshold, perform a corner curling detection process on the display image to obtain the corner curling detection result of the first object.

[0160] Step 1011: When the corner curling detection result indicates that the first object has a corner curl, display the corner curl adjustment prompt message.

[0161] The operation prompt message includes the corner curl adjustment prompt message.

[0162] Step 1012: When the corner curling detection result indicates that the first object does not have a corner curl, perform a motion detection process on the first object to obtain the motion detection result.

[0163] Step 1013: If the motion detection result indicates that the first object is in a motion state, display the static prompt message.

[0164] Step 1014: Obtain the next image to be processed. And start executing from the above step 1001 again.

[0165] Step 1015: If the motion detection result indicates that the first object is in a static state, determine the display image as the target image.

[0166] Step 1016: Send the target image into the target information recognition process to obtain the information recognition result corresponding to the target image.

[0167] A typical application scenario of the embodiment of the present application is a smart education scenario. In a possible implementation manner, the technical solution provided by the embodiment of the present application can be applied to the function of photographing and correcting homework in a smart education desk lamp. Please refer to Figure 11 , which exemplarily shows a flowchart of photographing and correcting homework. Figure 11 The shown process is as follows:

[0168] s1. Turn on the camera.

[0169] The smart education desk lamp responds to the target operation to enter the homework photographing process and turns on the camera.

[0170] Optionally, set the viewfinder area in the middle (the photographing range of the desk lamp camera is a rectangular area directly below).

[0171] s2. Regularly and cyclically capture images from the camera at a certain frame rate.

[0172] Optionally, the frame rate is set to 15 frames per second.

[0173] s3. Obtain the image to be processed from the camera.

[0174] After obtaining the image, first adjust the image resolution of the image to be processed from 2560*1920 to 640*480, aiming to reduce the time consumption of subsequent image detection.

[0175] s4. Display the image to be processed in the preview area of the desk lamp screen.

[0176] s5. The adjusted image is input into the edge detection process to obtain the four vertices of the exercise book.

[0177] Obtain the coordinates of the four vertices through the edge detection algorithm. The algorithm will return the coordinate data and the confidence level of each point. If the confidence level is too low, the vertex is considered not detected.

[0178] s6. Judge whether the coordinates of the four vertices are complete. If not, execute s7; if complete, execute s10.

[0179] Judge whether all four vertices are detected. If not, automatically move the camera viewfinder a fixed pixel value in the direction of the vertex not included in the preview interface to adjust the viewfinder area and detect the image to be processed captured by the camera again until all four vertices are within the viewfinder area or have reached the edge of the desk lamp's photo-taking range and cannot be moved anymore.

[0180] s7. Judge whether the viewfinder area has reached the edge of the camera shooting range and cannot be moved. If the viewfinder area cannot be moved anymore, execute s8; if the viewfinder area can continue to move, execute s9.

[0181] s8. Prompt the user: "The complete exercise book has not been detected. Please adjust the position."

[0182] If the viewfinder area cannot be moved anymore but the four vertices are still incomplete, voice prompt the user: "The complete exercise book has not been detected. Please adjust the position."

[0183] s9. Move the camera viewfinder area in the direction where the vertex is missing. Then start executing from s3.

[0184] Optionally, automatically move the camera viewfinder a fixed pixel value in the direction of the vertex not included in the preview interface.

[0185] s10. Scale the image to be processed based on the four vertices to display the exercise book at the maximum resolution.

[0186] Make the exercise book display as large as possible in the screen preview area.

[0187] If all four vertices are already within the viewfinder, a rectangle (the positioning area) can be determined, with each point on one side. Scale the image to be processed so that this rectangle is as large as possible and display it in the preview area. Optionally, the distance from each side of the rectangle to the edge of the preview area of the screen is 20 pixels.

[0188] S11. Determine whether the exercise book is placed correctly based on the vertex coordinates. If not, execute S12; if so, execute S14.

[0189] Determine whether the current orientation of the exercise book is correct based on the vertex coordinates. To improve flexibility, the angle error is set to be around 10 degrees here.

[0190] S12. Prompt the user: "The orientation is incorrect. Please place the exercise book upright."

[0191] If the angle is incorrect, voice prompt "The orientation is incorrect. Please place the exercise book upright."

[0192] S13. Draw a red edge border in the preview area.

[0193] When the angle is incorrect, draw a red border for the exercise book on the preview interface.

[0194] When there is a corner curl, draw a red border for the exercise book on the preview interface.

[0195] S14. Determine whether there is a corner curl in the exercise book. If there is a corner curl, execute S15; if normal, execute S16.

[0196] Optionally, use the ellipse detection algorithm to determine whether there is a corner curl on each edge of the exercise book.

[0197] S15. Prompt the user: "The exercise book is curled. Please ensure that all four corners are intact." And execute S13.

[0198] If there is a corner curl, voice prompt "The exercise book is curled. Please ensure that all four corners are intact."

[0199] S16. Prompt the user: "The placement is correct. Please stay still."

[0200] Voice prompt the user "The placement is correct. Please stay still." Optionally, draw a green border for the exercise book on the preview interface.

[0201] S17. Determine whether the exercise book is moving. If it is stationary, execute S18; if it is moving, execute S3.

[0202] Detect and judge whether the exercise book is moving or stationary based on the previously input image as a reference.

[0203] If in a moving state, no other processing is done and wait for the next camera image capture.

[0204] s18. Upload the homework image for background marking.

[0205] If the exercise book is in a stationary state, the original captured image is automatically compressed into a JPG (Joint Photographic Experts Group) image with a quality of 90% and uploaded to the background for homework marking.

[0206] In Figure 11 In the process of taking pictures, marking, and uploading homework shown, the device can, during the process of students moving and placing the exercise book, use an edge detection algorithm to track the position change of the exercise book in real time, making the exercise book always display at the center of the screen preview at the maximum resolution. Also, it can detect in real time whether the exercise book is placed correctly in the preview and draw border boxes of different colors to prompt the student to make corresponding adjustments. For example, when the exercise book is not in the correct orientation or is curled, a red prompt box is drawn; when it is placed correctly, a green box is drawn to indicate correct placement. When the device detects that the exercise book has changed from a moving state to a stationary state and is placed correctly, it can automatically take a picture and upload the homework image.

[0207] After applying the image processing method provided by the embodiments of this application, the interaction method of the process of taking pictures, marking, and uploading homework is user-friendly. The student's hand can always remain on the position of the exercise book without the need to separately click the take picture or upload button. The system will automatically detect the picture and automatically take a picture and complete the upload when the conditions are met. In addition, through some pre-detection and assistance in the embodiments of this application, students can be guided to take high-quality pictures, reducing the uncertainty caused by students' exploratory operations, ensuring that the homework pictures uploaded to the cloud have the maximum resolution, and the orientation and quality of the homework placement can meet the requirements of marking.

[0208] In summary, the technical solution provided by the embodiments of the present application can determine the edge information of the first object by detecting and processing the first object in the image to be processed; when the first object is incomplete, without the user manually adjusting the placement position of the first object, the device automatically moves the camera viewfinder in the direction of the missing edge to facilitate collecting an image including the complete first object, effectively reducing the operation complexity of image processing; after collecting an image including the complete first object, the positioning area where the first object is located in the image to be processed can be determined according to the edge information of the first object, and the image of the positioning area is displayed, effectively reducing the irrelevant content in the displayed image and effectively improving the image quality; in addition, the placement state of the first object in the displayed image is also detected, and an operation prompt can be sent to the user when the placement state is not good to prompt the user to adjust the placement state of the first object; when the placement state meets the preset conditions, the displayed image is determined as the target image and sent to the subsequent information recognition process, effectively improving the image quality, reducing the operation complexity of image processing, improving the overall efficiency of image processing, and optimizing the user experience.

[0209] Please refer to Figure 12 , which shows the flow of the image processing method provided by an embodiment of the present application Figure 4 . This method can be applied to a computer device, which refers to an electronic device with data calculation and processing capabilities. For example, the execution subject of each step can be Figure 2 the terminal 10 in the application program running environment shown. This method can include the following steps (1201-1209).

[0210] Step 1201, after obtaining each frame of the image to be processed, perform a second object detection process on the image to be processed to obtain a second detection result.

[0211] In an exemplary embodiment, the above second object is a user gesture. Sometimes, the results of the edge detection or vertex detection performed in the above embodiments are interfered by irrelevant content in the image. At this time, the position of the student's finger can be detected in real time through the camera, and the student triggers a gesture to determine the edge information or vertex position of the detection object. For example, in the intelligent education scenario, the detection of the vertices of the exercise book is sometimes inaccurate due to the interference of items on the table. At this time, the position of the student's finger can be detected in real time through the camera, and the student triggers a gesture to determine the four corner positions of the exercise book.

[0212] In a possible implementation manner, for the detection and recognition of the above second object, a second object detection model can be trained separately. Optionally, the above second object detection model is a machine learning model, the training sample is an image including a sample user gesture, and the sample label is the pixel point type, such as whether the pixel point type is a user gesture.

[0213] In this way, after each frame of the image to be processed is obtained, the image to be processed is input into the second object detection model for second object detection processing to obtain a second detection result.

[0214] The above second detection result is used to indicate whether the image to be processed includes a second object. In the case of whether the image to be processed includes a second object, the above second detection result includes the coordinates of the positioning points corresponding to the second object.

[0215] Step 1202: If the second detection results corresponding to n consecutive frames of the images to be processed indicate that the n consecutive frames of the images to be processed include a second object, determine the sliding trajectory of the second object among the n consecutive frames of the images to be processed.

[0216] The above n is a positive integer greater than or equal to 2.

[0217] In an exemplary embodiment, the sliding trajectory of the second object among the n consecutive frames of the images to be processed can be determined according to the coordinates of the positioning points corresponding to the second object in the n consecutive frames of the images to be processed. Optionally, the above sliding trajectory is formed by connecting the positioning points corresponding to the second object in the n consecutive frames of the images to be processed.

[0218] If the sliding trajectory meets the preset trajectory condition, it is determined that the n consecutive frames of the images to be processed include a first object, and the sliding trajectory is used to locate the first object. Optionally, the preset trajectory condition includes that the sliding trajectory of the second object matches the diagonal line or the contour line corresponding to the object shape of the first object. For example, the user slides a finger along the diagonal direction of an exercise book or along the contour of the exercise book to obtain the above sliding trajectory.

[0219] When the user makes a gesture sliding action, it also means that the n consecutive frames of the images to be processed include a first object.

[0220] Step 1203: When the sliding trajectory meets the preset trajectory condition, determine a positioning area based on the sliding trajectory.

[0221] After the system detects the above sliding trajectory that meets the preset trajectory condition, a positioning area can be determined in the first image to be processed according to the sliding trajectory, and this positioning area is the area where the first object is located in the first image to be processed. Optionally, the above first image to be processed is any one of the n consecutive frames of the images to be processed, or it can also be an image after the n consecutive frames of the images to be processed.

[0222] Step 1204: Display the display image corresponding to the positioning area.

[0223] The image corresponding to the positioning area, that is, the above display image, is displayed in the preview area of the target page to achieve the tracking display of the first object.

[0224] Step 1205: Determine the placement state of the first object in the displayed image.

[0225] Step 1206: Based on the placement state, display operation prompt information.

[0226] In an exemplary embodiment, determine the placement angle of the first object in the displayed image.

[0227] When the placement angle is greater than the angle threshold, display angle adjustment prompt information. The operation prompt information includes the angle adjustment prompt information.

[0228] When the placement angle is less than or equal to the angle threshold, perform a corner curling detection process on the displayed image to obtain the corner curling detection result of the first object.

[0229] When the corner curling detection result indicates that the first object has a corner curl, display corner curling adjustment prompt information. The operation prompt information includes the corner curling adjustment prompt information.

[0230] When the corner curling detection result indicates that the first object does not have a corner curl, perform a motion detection process on the first object to obtain a motion detection result.

[0231] If the motion detection result indicates that the first object is in a motion state, display a stillness prompt information.

[0232] Step 1207: When the placement state of the first object meets the preset conditions, determine the displayed image as the target image corresponding to the first object.

[0233] If the motion detection result indicates that the first object is in a stationary state, determine the displayed image as the target image.

[0234] Step 1208: Send the target image into the target information recognition process to obtain the information recognition result corresponding to the target image.

[0235] In summary, the technical solution provided in the embodiment of the present application can determine the sliding trajectory of the second object by performing a detection process on the second object in the image to be processed, and then can locate the first object according to the sliding trajectory of the second object, determine the positioning area where the first object is located, and display the image of the positioning area. In addition, it also detects the placement state of the first object in the displayed image, determines the displayed image as the target image under the condition of meeting the preset conditions and sends it into the subsequent information recognition process, improving the image quality, reducing the complexity of the image processing operation, and improving the overall efficiency of the image processing.

[0236] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For the details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the present application.

[0237] Please refer to Figure 13 which shows a block diagram of an image processing apparatus provided in an embodiment of the present application. The apparatus has the function of implementing the above-mentioned image processing method, and the function can be implemented by hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be provided in a computer device. The apparatus 1300 may include: an object detection module 1310, an information display module 1320, and an image determination module 1330.

[0238] The object detection module 1310 is configured to detect whether the to-be-processed image includes a first object after each frame of to-be-processed image is acquired.

[0239] The information display module 1320 is configured to display a display image corresponding to the first object and operation prompt information corresponding to the display image when it is detected that the first to-be-processed image includes the first object; wherein, the display image is an image corresponding to a positioning area where the first object is located in the first to-be-processed image, and the operation prompt information is used to prompt a target object to adjust the placement state of the first object.

[0240] The image determination module 1330 is configured to determine the display image as a target image corresponding to the first object when the placement state of the first object meets a preset condition.

[0241] In an exemplary embodiment, the information display module 1320 includes: an object positioning unit, an image display unit, and an information prompt unit.

[0242] The object positioning unit is configured to, if the first to-be-processed image includes the overall edge information of the first object, determine the positioning area and the placement angle of the first object based on the overall edge information.

[0243] The image display unit is configured to display the display image corresponding to the positioning area.

[0244] The information prompt unit is configured to display angle adjustment prompt information when the placement angle is greater than an angle threshold, and the operation prompt information includes the angle adjustment prompt information.

[0245] In an exemplary embodiment, the information display module 1320 further includes: a corner curling detection unit.

[0246] The corner curling detection unit is configured to perform corner curling detection processing on the display image to obtain a corner curling detection result of the first object.

[0247] The information prompting unit is further configured to display a corner curling adjustment prompt message when the corner curling detection result indicates that the first object has a corner curling, and the operation prompt message includes the corner curling adjustment prompt message.

[0248] In an exemplary embodiment, the image determination module 1330 includes: a motion detection unit and an image determination unit.

[0249] The motion detection unit is configured to perform motion detection processing on the first object to obtain a motion detection result when the placement angle is less than or equal to the angle threshold and the corner curling detection result indicates that the first object does not have a corner curling.

[0250] The image determination unit is configured to determine the display image as the target image if the motion detection result indicates that the first object is in a stationary state.

[0251] In an exemplary embodiment, the information display module 1320 includes: a local object positioning unit and an information display unit.

[0252] The local object positioning unit is configured to determine the positioning area based on the local edge information if the first image to be processed includes the local edge information of the first object.

[0253] The information display unit is configured to display the display image corresponding to the positioning area and the position adjustment prompt message corresponding to the local edge information, and the operation prompt message includes the position adjustment prompt message.

[0254] In an exemplary embodiment, the apparatus 1300 further includes: a viewfinder direction determination module, a viewfinder movement module, and an image acquisition module.

[0255] The viewfinder direction determination module is configured to determine the edge missing direction corresponding to the first object in the second image to be processed according to the local edge information when it is detected that the second image to be processed includes the local edge information of the first object.

[0256] The viewfinder movement module is configured to move the image viewfinder in the camera shooting area from a first position to a second position according to the edge missing direction; wherein, the first position is the current position of the image viewfinder, and the second position is the position after the image viewfinder is moved.

[0257] The image acquisition module is configured to perform image acquisition on the camera shooting area when the image viewfinder is located at the second position, re-acquire the image to be processed, and start executing the step of detecting whether the image to be processed includes the first object again after each frame of the image to be processed is acquired.

[0258] In an exemplary embodiment, the local object positioning unit is further configured to, when it is detected that the first image to be processed includes local edge information of the first object and the image viewfinder reaches the edge of the camera shooting area, determine the positioning area based on the local edge information;

[0259] The information display unit is configured to display the display image corresponding to the positioning area and the position adjustment prompt information corresponding to the local edge information;

[0260] Wherein, the first image to be processed is an image collected when the image viewfinder reaches the edge of the area, and the operation prompt information includes the position adjustment prompt information.

[0261] In an exemplary embodiment, the object detection module 1310 includes: a first object detection unit and a first object determination unit.

[0262] The first object detection unit is configured to perform a first object detection process on the image to be processed to obtain a first detection result;

[0263] The first object determination unit is configured to, if the first detection result corresponding to the first image to be processed indicates that the first image to be processed includes the overall edge information or local edge information of the first object, determine that the first image to be processed includes the first object.

[0264] In an exemplary embodiment, the object detection module 1310 further includes: a second object detection unit and a sliding trajectory determination unit.

[0265] The second object detection unit is configured to perform a second object detection process on the image to be processed to obtain a second detection result;

[0266] The sliding trajectory determination unit is configured to, if the second detection results corresponding to n consecutive frames of images to be processed indicate that the n consecutive frames of images to be processed include the second object, determine the sliding trajectory of the second object between the n consecutive frames of images to be processed, where n is a positive integer greater than or equal to 2;

[0267] The first object determination unit is further configured to, if the sliding trajectory meets a preset trajectory condition, determine that the n consecutive frames of images to be processed include the first object, and the sliding trajectory is used to locate the first object.

[0268] In an exemplary embodiment, the information display module 1320 further includes: a sliding trajectory positioning unit and an object placement state determination unit.

[0269] A sliding trajectory positioning unit, configured to determine the positioning area based on the sliding trajectory when the sliding trajectory meets the preset trajectory condition;

[0270] The image display unit is further configured to display a display image corresponding to the positioning area;

[0271] An object placement state determination unit, configured to determine the placement state of the first object in the display image;

[0272] The information prompt unit is further configured to display the operation prompt information based on the placement state.

[0273] In an exemplary embodiment, the apparatus 1300 further includes: an information recognition module.

[0274] The information recognition module is configured to send the target image into a target information recognition process to obtain an information recognition result corresponding to the target image.

[0275] In summary, the technical solution provided in the embodiment of the present application can quickly detect a to-be-processed image containing a first object after obtaining a frame of to-be-processed image; when it is detected that the first to-be-processed image contains the first object, a partial image corresponding to the positioning area where the first object is located in the to-be-processed image will be displayed, effectively reducing the image content irrelevant to the first object in the to-be-processed image and improving the image quality; and corresponding operation prompt information will also be displayed to prompt the user to timely adjust the placement state of the first object, and the display image can be determined as the target image when the placement state meets the preset conditions, reducing the complexity of the image processing operation, improving the overall efficiency of the image processing, and optimizing the user experience.

[0276] It should be noted that, when implementing its functions, the apparatus provided in the above embodiment is only illustrated by dividing the above functional modules. In actual application, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus provided in the above embodiment and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0277] Please refer to Figure 14 , which shows a structural block diagram of a computer device provided in an embodiment of the present application. The computer device may be a terminal. The computer device is used to implement the image processing method provided in the above embodiment. Specifically:

[0278] Generally, the computer device 1400 includes: a processor 1401 and a memory 1402.

[0279] The processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The processor 1401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1401 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 1401 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1401 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0280] The memory 1402 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 1402 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1402 is used to store at least one instruction, at least one segment of program, code set, or instruction set, and the at least one instruction, at least one segment of program, code set, or instruction set is configured to be executed by one or more processors to implement the above image processing method.

[0281] In some embodiments, the computer device 1400 may also optionally include: a peripheral device interface 1403 and at least one peripheral device. The processor 1401, the memory 1402, and the peripheral device interface 1403 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 1403 through a bus, signal lines, or a circuit board. Specifically, the peripheral devices include at least one of a radio frequency circuit 1404, a touch display screen 1405, a camera component 1406, an audio circuit 1407, a positioning component 1408, and a power supply 1409.

[0282] Those skilled in the art can understand, Figure 14The structure shown does not limit the computer device 1400, which may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement, such as Figure 15 , Figure 15 is a structural block diagram of a computer device provided by another embodiment of the present application.

[0283] In an exemplary embodiment, the above computer device is a point-reading device. Optionally, the above point-reading device includes the above intelligent education desk lamp.

[0284] In an exemplary embodiment, a computer-readable storage medium is further provided. At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium. When the at least one instruction, the at least one program, the code set, or the instruction set is executed by a processor, the above image processing method is implemented.

[0285] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or an optical disc, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0286] In an exemplary embodiment, a computer program product or a computer program is further provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above image processing method.

[0287] It should be understood that "a plurality of" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. In addition, the step numbers described herein only exemplarily show a possible execution sequence between steps. In some other embodiments, the above steps may not be executed in the order of the numbers. For example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in the reverse order of the illustration. The embodiments of the present application do not limit this.

[0288] In addition, in the specific implementation manners of the present application, data related to user information and the like are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0289] The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: After obtaining each frame of the image to be processed, detecting whether the image to be processed includes a first object; the first object is the object to be photographed; When it is detected that the first image to be processed includes the first object, displaying a display image corresponding to the first object and operation prompt information corresponding to the display image; wherein, the display image is an image corresponding to the positioning area where the first object is located in the first image to be processed, and the operation prompt information is used to prompt the target object to adjust the placement state of the first object; the positioning area is a partial area of the first image to be processed that includes the first object, the image area occupied by the first object in the display image is greater than or equal to a preset ratio threshold, and the image content outside the positioning area in the first image to be processed is content image unrelated to the first object; When the placement state of the first object meets the preset conditions, determining the display image as the target image corresponding to the first object; Sending the target image into a target information recognition process to obtain an information recognition result corresponding to the target image.

2. The method according to claim 1, characterized in that The step of, when it is detected that the first image to be processed includes the first object, displaying a display image corresponding to the first object and operation prompt information corresponding to the display image includes: If the first image to be processed includes the overall edge information of the first object, determining the positioning area and the placement angle of the first object based on the overall edge information; Displaying the display image corresponding to the positioning area; When the placement angle is greater than the angle threshold, displaying angle adjustment prompt information, and the operation prompt information includes the angle adjustment prompt information.

3. The method according to claim 2, wherein The method further includes: Performing a corner curling detection process on the display image to obtain a corner curling detection result of the first object; When the corner curling detection result indicates that the first object has a corner curl, displaying corner curling adjustment prompt information, and the operation prompt information includes the corner curling adjustment prompt information.

4. The method according to claim 3, wherein The step of, when the placement state of the first object meets the preset conditions, determining the display image as the target image corresponding to the first object includes: When the placement angle is less than or equal to the angle threshold and the corner curling detection result indicates that the first object does not have a corner curl, performing a motion detection process on the first object to obtain a motion detection result; If the motion detection result indicates that the first object is in a stationary state, determining the display image as the target image.

5. The method according to claim 1, characterized in that The step of, when it is detected that the first image to be processed includes the first object, displaying a display image corresponding to the first object and operation prompt information corresponding to the display image includes: If the first image to be processed includes the partial edge information of the first object, determining the positioning area based on the partial edge information; Displaying the display image corresponding to the positioning area and position adjustment prompt information corresponding to the partial edge information, and the operation prompt information includes the position adjustment prompt information.

6. The method according to claim 1, wherein The method further includes: When it is detected that the second image to be processed includes the local edge information of the first object, determine the edge missing direction corresponding to the first object in the second image to be processed according to the local edge information; According to the edge missing direction, move the image viewfinder in the camera shooting area from the first position to the second position; wherein, the first position is the current position of the image viewfinder, and the second position is the position after the image viewfinder is moved; When the image viewfinder is located at the second position, perform image acquisition on the camera shooting area, re-acquire the image to be processed, and start executing the step of detecting whether the image to be processed includes the first object again after each frame of the image to be processed is acquired.

7. The method according to claim 6, wherein The step of, when it is detected that the first image to be processed includes the first object, displaying the display image corresponding to the first object and the operation prompt information corresponding to the display image includes: When it is detected that the first image to be processed includes the local edge information of the first object and the image viewfinder reaches the area edge of the camera shooting area, determine the positioning area based on the local edge information; Display the display image corresponding to the positioning area and the position adjustment prompt information corresponding to the local edge information; Wherein, the first image to be processed is the image acquired when the image viewfinder reaches the area edge, and the operation prompt information includes the position adjustment prompt information.

8. The method according to claim 1, characterized in that The step of detecting whether the image to be processed includes the first object includes: Perform the first object detection process on the image to be processed to obtain the first detection result; If the first detection result corresponding to the first image to be processed indicates that the first image to be processed includes the overall edge information or local edge information of the first object, determine that the first image to be processed includes the first object.

9. The method according to claim 1, characterized in that The step of detecting whether the image to be processed includes the first object includes: Perform the second object detection process on the image to be processed to obtain the second detection result; If the second detection results corresponding to the continuous n frames of images to be processed indicate that the continuous n frames of images to be processed include the second object, determine the sliding trajectory of the second object between the continuous n frames of images to be processed, where n is a positive integer greater than or equal to 2; If the sliding trajectory meets the preset trajectory condition, determine that the continuous n frames of images to be processed include the first object, and the sliding trajectory is used to locate the first object.

10. The method according to claim 9, characterized in that, The step of, when it is detected that the first image to be processed includes the first object, displaying the display image corresponding to the first object and the operation prompt information corresponding to the display image includes: When the sliding trajectory meets the preset trajectory condition, determine the positioning area based on the sliding trajectory; Display the display image corresponding to the positioning area; Determine the placement state of the first object in the display image; Based on the placement state, display the operation prompt information.

11. An image processing apparatus, characterized in that, The device includes: An object detection module, configured to detect whether a to-be-processed image includes a first object after each frame of to-be-processed image is acquired; the first object is a photographed object; An information display module, configured to display a display image corresponding to the first object and operation prompt information corresponding to the display image when it is detected that the first to-be-processed image includes the first object; wherein, the display image is an image corresponding to a positioning area where the first object is located in the first to-be-processed image, and the operation prompt information is used to prompt a target object to adjust the placement state of the first object; the positioning area is a partial area of the first to-be-processed image that includes the first object, the image area occupied by the first object in the display image is greater than or equal to a preset ratio threshold, and the image content outside the positioning area in the first to-be-processed image is content image unrelated to the first object; An image determination module, configured to determine the display image as the target image corresponding to the first object when the placement state of the first object meets a preset condition; An information recognition module, configured to send the target image into a target information recognition process to obtain an information recognition result corresponding to the target image.

12. The device according to claim 11, characterized in that, The information display module includes: An object positioning unit, configured to determine the positioning area and the placement angle of the first object based on the overall edge information if the first to-be-processed image includes the overall edge information of the first object; An image display unit, configured to display the display image corresponding to the positioning area; An information prompt unit, configured to display angle adjustment prompt information when the placement angle is greater than an angle threshold, and the operation prompt information includes the angle adjustment prompt information.

13. The device according to claim 12, characterized in that, The information display module further includes: A corner curling detection unit, configured to perform corner curling detection processing on the display image to obtain a corner curling detection result of the first object; The information prompt unit is further configured to display corner curling adjustment prompt information when the corner curling detection result indicates that the first object has a corner curling, and the operation prompt information includes the corner curling adjustment prompt information.

14. The device according to claim 13, characterized in that, The image determination module includes: A motion detection unit, configured to perform motion detection processing on the first object to obtain a motion detection result when the placement angle is less than or equal to the angle threshold and the corner curling detection result indicates that the first object has no corner curling; An image determination unit, configured to determine the display image as the target image if the motion detection result indicates that the first object is in a stationary state.

15. The device according to claim 11, characterized in that, The information display module includes: A local object positioning unit, configured to determine the positioning area based on the local edge information if the first to-be-processed image includes the local edge information of the first object; An information display unit, configured to display the display image corresponding to the positioning area and position adjustment prompt information corresponding to the local edge information, and the operation prompt information includes the position adjustment prompt information.

16. The device according to claim 11, characterized in that, The device further includes: A viewing direction determination module, configured to, when it is detected that the second image to be processed includes local edge information of the first object, determine an edge missing direction corresponding to the first object in the second image to be processed according to the local edge information; A viewfinder moving module, configured to move an image viewfinder in a camera shooting area from a first position to a second position according to the edge missing direction; wherein, the first position is the current position of the image viewfinder, and the second position is the position after the image viewfinder is moved; An image acquisition module, configured to, when the image viewfinder is located at the second position, perform image acquisition on the camera shooting area, re-acquire the image to be processed, and start executing again from the step of detecting whether the image to be processed includes the first object after each frame of the image to be processed is acquired.

17. The device according to claim 16, characterized in that, The information display module includes: A local object positioning unit, configured to, when it is detected that the first image to be processed includes local edge information of the first object and the image viewfinder reaches the area edge of the camera shooting area, determine the positioning area based on the local edge information; An information display unit, configured to display the display image corresponding to the positioning area and the position adjustment prompt information corresponding to the local edge information; Wherein, the first image to be processed is an image acquired when the image viewfinder reaches the area edge, and the operation prompt information includes the position adjustment prompt information.

18. The device according to claim 11, characterized in that, The object detection module includes: a first object detection unit, configured to perform first object detection processing on the image to be processed to obtain a first detection result; A first object determination unit, configured to, if the first detection result corresponding to the first image to be processed indicates that the first image to be processed includes the overall edge information or local edge information of the first object, determine that the first image to be processed includes the first object.

19. The device according to claim 11, wherein, The object detection module includes: A second object detection unit, configured to perform second object detection processing on the image to be processed to obtain a second detection result; A sliding trajectory determination unit, configured to, if the second detection results corresponding to consecutive n frames of images to be processed indicate that the consecutive n frames of images to be processed include the second object, determine a sliding trajectory of the second object between the consecutive n frames of images to be processed, where n is a positive integer greater than or equal to 2; A first object determination unit, configured to, if the sliding trajectory meets a preset trajectory condition, determine that the consecutive n frames of images to be processed include the first object, and the sliding trajectory is used to locate the first object.

20. The device according to claim 19, characterized in that, The information display module includes: A sliding trajectory positioning unit, configured to, when the sliding trajectory meets the preset trajectory condition, determine the positioning area based on the sliding trajectory; An image display unit, configured to display the display image corresponding to the positioning area; An object placement state determination unit, configured to determine a placement state of the first object in the display image; An information prompt unit, configured to display the operation prompt information based on the placement state.

21. A computer device, characterized in that, The computer device includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image processing method according to any one of claims 1 to 10.

23. A computer program product, characterized in that, The computer program product includes computer instructions. The computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes to implement the image processing method according to any one of claims 1 to 10.

Images