Photo search method, device, equipment and storage medium

Abstract

The present application discloses a method, device, equipment and storage medium for searching questions by taking photos, and relates to artificial intelligence fields such as natural language processing and computer vision. A specific implementation of the method includes: photographing and uploading a multi-question image, wherein the multi-question image includes multiple exercises; receiving the coordinates of multiple exercises obtained by performing multi-question detection on the multi-question image; selecting the multi-question image based on the coordinates of the multiple exercises to obtain selection frames of the multiple exercises; uploading the selection frame of at least one exercise among the multiple exercises; and receiving the search results obtained by searching for questions based on at least one exercise. This implementation supports multi-question recognition and automatic question framing, simplifies the operation of users when searching for questions by taking photos, optimizes the interactive experience of searching for questions by taking photos, and makes searching for questions by taking photos more convenient.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, specifically to artificial intelligence fields such as natural language processing and computer vision, and especially to a method, device, equipment and storage medium for searching questions by taking photos. Background Art

[0002] At present, in order to solve the problem of difficult homework for students, various photo search applications have emerged. Existing photo search applications all allow users to take a photo of a single exercise, submit a picture of the single exercise for search, and finally return the search results. The photo search function provided by the photo search application can only search for one exercise at a time. If the user wants to continue searching for other exercises after searching for one exercise, he needs to take another photo and search again. Summary of the invention

[0003] The embodiments of the present application provide a method, device, equipment and storage medium for searching questions by taking photos.

[0004] In a first aspect, an embodiment of the present application proposes a method for searching questions by taking photos, comprising: photographing and uploading a multi-question image, wherein the multi-question image includes a plurality of exercises; receiving coordinates of a plurality of exercises obtained by performing a multi-question detection on the multi-question image; selecting the multi-question image based on the coordinates of the plurality of exercises to obtain selection frames of the plurality of exercises; uploading a selection frame of at least one exercise among the plurality of exercises; and receiving a search result obtained by searching for questions based on at least one exercise.

[0005] In the second aspect, an embodiment of the present application proposes a photo-taking question-searching device, comprising: a first uploading module, configured to take and upload a multi-question image, wherein the multi-question image includes a plurality of exercises; a first receiving module, configured to receive the coordinates of a plurality of exercises obtained by performing a multi-question detection on the multi-question image; a frame selection module, configured to frame the multi-question image based on the coordinates of the plurality of exercises to obtain a selection frame of the plurality of exercises; a second uploading module, configured to upload a selection frame of at least one exercise among the plurality of exercises; and a second receiving module, configured to receive a search result obtained by searching for questions based on at least one exercise.

[0006] In a third aspect, an embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method described in any implementation manner in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute a method as described in any implementation manner in the first aspect.

[0008] In a fifth aspect, an embodiment of the present application proposes a computer program product, including a computer program, which implements the method described in any implementation manner in the first aspect when executed by a processor.

[0009] The photo search method, device, equipment and storage medium provided in the embodiments of the present application first take and upload a multi-question image including multiple exercises, and receive the coordinates of the multiple exercises obtained by performing multi-question detection on the multi-question image; then frame the multi-question image based on the coordinates of the multiple exercises to obtain a selection frame of the multiple exercises; finally upload a selection frame of at least one exercise among the multiple exercises, and receive the search result obtained by searching for questions based on at least one exercise. Photo search supports multi-question recognition and automatic question framing, which does not require users to take photos repeatedly or manually frame questions, thus simplifying the user's operation when searching for questions by taking photos, optimizing the interactive experience of searching for questions by taking photos, and making searching for questions by taking photos more convenient. Photo search can be applied to smart learning tablets to increase the core competitiveness of smart learning tablets.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, purposes and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are used to better understand the present solution and do not constitute a limitation of the present application. Among them:

[0012] Figure 1 is a flow chart of an embodiment of the method for searching questions by taking photos according to the present application;

[0013] Figure 2A It is a schematic diagram of the title page;

[0014] Figure 2B This is a schematic diagram of the scanning animation floating layer;

[0015] Figure 2C This is a schematic diagram of the frame selection coordinate display page;

[0016] Figure 2D This is a schematic diagram of the results page;

[0017] Figure 3 It is a flow chart of the interaction of taking photos and searching questions;

[0018] Figure 4 is a flow chart of another embodiment of the method for searching questions by taking photos according to the present application;

[0019] Figure 5is a flow chart of another embodiment of the method for searching questions by taking photos according to the present application;

[0020] Fig. 6A is a schematic diagram of the first guide information;

[0021] Figure 6B is a schematic diagram of the second guide information;

[0022] Figure 6C is a schematic diagram of the third guide information;

[0023] Fig.6D is a schematic diagram of the fourth guide information;

[0024] Fig. 6E is a schematic diagram of the fifth guide information;

[0025] Figure 7 is a structural schematic diagram of an embodiment of a photo-taking question-searching device according to the present application;

[0026] Figure 8 It is a block diagram of an electronic device used to implement the photo-taking and question-searching method of an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] Figure 1 A process 100 of an embodiment of a method for searching questions by taking photos according to the present application is shown. The method for searching questions by taking photos comprises the following steps:

[0030] Step 101, take and upload multi-topic images.

[0031] In this embodiment, the client can take a multi-question image and upload it to the server, wherein the multi-question image may include multiple exercises.

[0032] Usually, a photo search application is installed on the client. When the user opens the photo search application, a photo search page can be displayed. The photo search application here not only supports single-question image capture, but also supports multi-question image capture. The user can switch the photo search mode on the photo search application's photo search page. Figure 2A A schematic diagram of the title page is shown. Figure 2A As shown, the user clicks "Single Question" on the shooting page to switch to single question mode. The user clicks "Multiple Questions" on the shooting page to switch to multiple question mode. At the same time, it prompts "Multiple questions can be photographed at one time, and the text is aligned with the reference line." The user can place the exercise document in an appropriate position so that the exercise the user needs to photograph appears in the viewfinder of the shooting page. The user clicks the shooting button on the shooting page to capture the exercise image. During the process of shooting the exercise image, the viewfinder of the shooting page will scan the animation floating layer. Figure 2B A schematic diagram of the scanning animation floating layer is shown. After scanning is completed, the exercise image can be obtained. If there is only one exercise in the viewfinder, the user can switch to the single-question mode and click the shooting button. The exercise image captured at this time is the single-question image. If there are multiple exercises in the viewfinder, the user can switch to the multi-question mode and click the shooting button. The exercise image captured at this time is the multi-question image.

[0033] In actual applications, in addition to exercises, the exercise document may also include other contents such as exercise type, answer instructions, and exercise explanations. In some embodiments, in addition to the exercises, other contents are presented in the viewfinder. In this case, the captured exercise image includes other contents in addition to the exercises. For example, a multi-question image may include answer instructions and multiple exercises. For another example, a multi-question image may include multiple exercises and exercise explanations for at least some of the exercises.

[0034] In addition, the shooting page of the photo search application can also display shooting guidance information. Combined with the tablet's hardware gyroscope, it provides guidance and prompts for users in various usage scenarios. Even when the tablet's protective cover has multiple placement angles, it can also ensure that the shooting effect is optimal.

[0035] Step 102, receiving coordinates of multiple exercises obtained by performing multiple-question detection on the multiple-question image.

[0036] In this embodiment, the server can perform multi-question detection on the multi-question image, obtain the coordinates of each question in the multi-question image, and send them to the client. In this way, the client receives the coordinates of each question in the multi-question image. Among them, the coordinates of the question can determine the selection box of the question. The selection box of the question is the bounding box of the question. For example, the coordinates of the question can include the coordinates of at least three vertices of the selection box of the question.

[0037] Usually, when the server detects a multi-question image, it will identify whether the content in the multi-question image is an exercise. If it is an exercise, the server will detect the coordinates of the exercise. If it is not an exercise, the server can end the detection of this part of the content, or continue to process this part of the content. Among them, the processing method may include but is not limited to coordinate detection, content deletion, content annotation, content improvement, etc. For example, if the server recognizes the answer instructions in the multi-question image, it can end the detection of the answer instructions and will not output the coordinates of the answer instructions. For another example, if the server recognizes the exercise explanation in the multi-question image, it can not only detect the coordinates of the exercise explanation, but also identify the content of the exercise explanation, and further improve the content of the exercise explanation.

[0038] Step 103 , frame the multiple-question images based on the coordinates of the multiple exercises to obtain the selection frames of the multiple exercises.

[0039] In this embodiment, the client can select the multi-question image based on the coordinates of each question to obtain a selection box for each question, wherein the rectangle with the coordinates of the question as the vertex is the selection box of the question.

[0040] In some optional implementations of this embodiment, in order to facilitate users to search for questions, the selection box of each exercise is marked with a serial number. Figure 2C The figure shows a schematic diagram of the frame selection coordinate display page. Figure 2C As shown, there are three exercise selection boxes on the coordinate display page, numbered ①, ②, and ③ respectively.

[0041] Step 104 , uploading a selection frame of at least one exercise from a plurality of exercises.

[0042] In this embodiment, the client can upload a selection frame of at least one exercise from the multiple exercises. Specifically, the client can segment the selection frame of at least one exercise from the multiple exercise images and upload it to the server.

[0043] In some optional implementations of this embodiment, the client can upload the selection boxes of all exercises in the multi-question image. The shooting button is enhanced, and the question search can be performed by clicking the shooting button, and the search results of all exercises can be obtained without user selection.

[0044] In some optional implementations of this embodiment, the user can select a question box from multiple question boxes, and the client can segment the selected box from the multi-question image and upload it to the server. Search questions based on the user's selection, thereby achieving targeted search. Among them, the user can select the box by clicking on the question box. In addition, the preset area of ​​the shooting page can display information prompting the user to select the question box, and its content can be, for example, "The following questions are found, select the question you want to search for." The information is usually displayed in the upper right corner of the multi-question image and does not change with the click of the button in the page. After the user clicks on the question box, it will enter the loading state.

[0045] In some optional implementations of this embodiment, in order to ensure that the selection box includes the complete exercise content, the selection box supports manual fine-tuning. Specifically, when the user selects the selection box of any exercise, the selected selection box can be highlighted and an adjustment button will be presented on it. When the user operates the adjustment button, the selected selection box will be adjusted accordingly. Among them, the adjustment button can be dragged and stretched. When the user drags the adjustment button, the position of the selected selection box can be adjusted according to the user's dragging operation. When the user stretches the adjustment button, the size of the selected selection box can be adjusted according to the user's stretching operation.

[0046] Step 105: receiving a search result obtained by searching for questions based on at least one exercise.

[0047] In this embodiment, the server can search for questions based on the content in the received selection box, obtain the search results, and send them to the client. In this way, the client receives the search results.

[0048] Usually, the server can recognize the text content in the received selection box. For example, the selection box is recognized using OCR (Optical Character Recognition) technology to obtain the text content in the selection box. Subsequently, the server can search in the question bank based on the text content in the selection box to obtain the search result. Figure 2D A schematic diagram of the result page is shown. Figure 2D As shown in the figure, the result page shows four answers to the exercise: Answer 1, Answer 2, Answer 3 and Answer 4. Each answer includes the title, test point, analysis, solution and knowledge point explanation.

[0049] The photo search method provided in the embodiment of the present application first takes and uploads a multi-question image including multiple exercises, and receives the coordinates of the multiple exercises obtained by performing multi-question detection on the multi-question image; then frames the multi-question image based on the coordinates of the multiple exercises to obtain selection frames of the multiple exercises; finally, uploads the selection frame of at least one exercise among the multiple exercises, and receives the search results obtained by searching for questions based on at least one exercise. Photo search supports multi-question recognition and automatic question framing, which does not require users to take photos repeatedly or manually frame questions, thus simplifying the user's operation when searching for questions by taking photos, optimizing the interactive experience of searching for questions by taking photos, and making searching for questions by taking photos more convenient. Photo search can be applied to smart learning tablets to increase the core competitiveness of smart learning tablets.

[0050] Continue to refer Figure 3 , which shows the interactive flow chart of taking photos to search for questions. Figure 3 As shown, the user clicks the icon of the photo search application on the client, and the client sends a click request to aries, which sends launchAPPPlayload to the client. The user clicks the shooting button to take photos of multiple questions, and the client uploads the multiple questions to aries. Aries requests the multi-question interface from Multimode, and Multimode returns the multiple questions coordinates to aries, which returns the multiple questions coordinates to the client. The client selects the multiple questions images based on the coordinates to obtain the selection boxes of multiple exercises. The user clicks the selection box of an exercise, and the client takes a screenshot based on the coordinates of the exercise to obtain the selection box image of the exercise. The client uploads the selection box image to aries, and aries uploads the selection box image to Multimode. Multimode sends the corresponding search results to aries. Aries requests Swan to send the results. Swan sends the search results to the client.

[0051] Further references Figure 4 , which shows a process 400 of another embodiment of the method for searching questions by taking photos according to the present application. The method for searching questions by taking photos includes the following steps:

[0052] Step 401, in response to detecting a front shooting scene, displaying guidance information for installing an image acquisition device and placing a tablet.

[0053] In this embodiment, after entering the shooting page, the client can detect the shooting scene. In response to detecting the front shooting scene, the client can display guidance information for installing the image acquisition device and placing the tablet. Among them, the scene of turning on the front camera is the front shooting scene. The image acquisition device can be a front camera, also called a smart eye. The guidance information for installing the image acquisition device and placing the tablet can be a picture plus text. The user can install the image acquisition device according to the guidance information for installing the image acquisition device and placing the tablet, and place the tablet in the inner positioning groove of the protective cover (for example, the first card slot of the protective cover). Fig. 6A FIG. 4 shows a schematic diagram of the first guide information. Fig. 6A As shown, the first guidance information guides the user to install the smart eye and stand up the tablet.

[0054] Step 402: In response to the image acquisition device not being installed, displaying guidance information for installing the image acquisition device.

[0055] In this embodiment, if the image acquisition device is not installed, the client can display guidance information for installing the image acquisition device regardless of whether the tablet angle is correct. The guidance information for installing the image acquisition device can be a picture plus text. The user can install the image acquisition device according to the guidance information for installing the image acquisition device.

[0056] Step 403: In response to the image acquisition device being installed but the tablet angle being incorrect, displaying guidance information for placing the tablet.

[0057] In this embodiment, if the image acquisition device has been installed but the tablet angle is incorrect, the client can display guidance information for placing the tablet. The guidance information for placing the tablet can be a picture plus text. The user can place the tablet into the inner positioning groove of the protective cover according to the guidance information for placing the tablet. Figure 6C FIG. 4 shows a schematic diagram of the third guide information. Figure 6C As shown, the third guide information guides the user to put the smart eye in place and support the tablet in the first card slot.

[0058] Step 404 , in response to the image acquisition device being installed and the tablet angle being correct, displaying guidance information for placing the exercise document.

[0059] In this embodiment, if the image acquisition device has been installed and the tablet is at the correct angle, the client can display guidance information for placing the exercise document. The guidance information for placing the exercise document can be a picture plus text. The user can place the exercise document in the trapezoidal area mask image according to the guidance information for placing the exercise document, and align the text with the reference line. Usually, the exercise document is placed under the tablet, keeping the exercise centered and aligned with the bottom of the tablet, and the exercise document can be placed in the trapezoidal area mask image. Figure 6B FIG. 1 shows a schematic diagram of the second guide information. Figure 6B As shown, the second guide information guides the user to place the exercise document in the trapezoidal area and keep it horizontal.

[0060] In some optional implementations of this embodiment, the guidance information is usually displayed when the photo search function is first turned on, and its form is usually a GIF animation. In addition, in order to avoid blocking the user's sight by displaying for a long time, the guidance information can automatically fade out after displaying for a preset time (for example, 3 seconds).

[0061] Step 405, take and upload multi-topic images.

[0062] Step 406, receiving coordinates of multiple exercises obtained by performing multiple-question detection on the multiple-question image.

[0063] Step 407 , frame the multiple-question images based on the coordinates of the multiple exercises to obtain selection frames of the multiple exercises.

[0064] Step 408, uploading a selection frame of at least one exercise from the plurality of exercises.

[0065] Step 409, receiving a search result obtained by searching for questions based on at least one exercise.

[0066] In this embodiment, the specific operations of steps 405-409 are already described in Figure 1 In the illustrated embodiment, steps 101 - 105 are described in detail and will not be repeated here.

[0067] from Figure 4 It can be seen that Figure 1 Compared with the corresponding embodiment, the photo search method in this embodiment adds a step of displaying the guidance information of the front shooting scene. Therefore, the scheme described in this embodiment combines the hardware gyroscope of the tablet to guide and prompt the user's front shooting scene. In the case where the protective cover of the tablet has multiple placement angles, the user is guided to use the tablet and the posture of taking questions on the question shooting page. Guide the user to place the tablet correctly and put the question straight to ensure the best shooting effect.

[0068] Further references Figure 5 , which shows a process 500 of another embodiment of the method for searching questions by taking photos according to the present application. The method for searching questions by taking photos includes the following steps:

[0069] Step 501 , in response to detecting a post-shooting scene, displaying guidance information for placing a tablet and an exercise document.

[0070] In this embodiment, after entering the shooting page, the client can detect the shooting scene. In response to detecting the rear shooting scene, the client can display guidance information for placing the tablet and the exercise document. Among them, the scene of turning on the rear camera is the rear shooting scene. The guidance information for displaying the placement of the tablet and the exercise document can be a picture plus text. The user can place the tablet and the exercise document in parallel according to the guidance information for displaying the placement of the tablet and the exercise document, and place the exercise document in the tic-tac-toe masking layer picture, parallel to the exercise document paper, and align the text with the reference line. Fig.6D FIG. 4 shows a schematic diagram of the fourth guide information. Fig.6D As shown, the fourth guide information guides the user to place the tablet parallel to the exercise document. Fig. 6E FIG. 4 shows a schematic diagram of the fifth guide information. Fig. 6E As shown, the fifth guide information guides the user to place the exercise document in front of the tablet.

[0071] In some optional implementations of this embodiment, the guidance information is usually displayed when the photo search function is first turned on, and its form is usually a GIF animation. In addition, in order to avoid blocking the user's sight by displaying for a long time, the guidance information can automatically fade out after displaying for a preset time (for example, 3 seconds).

[0072] Step 502, take and upload multi-topic images.

[0073] Step 503, receiving coordinates of multiple exercises obtained by performing multiple-question detection on the multiple-question image.

[0074] Step 504 , frame the multiple-question images based on the coordinates of the multiple exercises to obtain selection frames of the multiple exercises.

[0075] Step 505: Upload a selection box of at least one exercise from a plurality of exercises.

[0076] Step 506, receiving a search result obtained by searching for questions based on at least one exercise.

[0077] In this embodiment, the specific operations of steps 502-506 are already Figure 1 In the illustrated embodiment, steps 101 - 105 are described in detail and will not be repeated here.

[0078] from Figure 5 It can be seen that Figure 1 Compared with the corresponding embodiment, the photo search method in this embodiment adds a step of displaying guidance information of the post-shooting scene. Therefore, the solution described in this embodiment combines the hardware gyroscope of the tablet to guide and prompt the user's post-shooting scene. Strengthen guidance on the user's use of the tablet and the posture of shooting questions on the shooting question page. Guide the user to place the tablet correctly and put the question straight to ensure the best shooting effect.

[0079] Further references Figure 7 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for searching questions by taking photos. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0080] like Figure 7As shown, the photographing and searching apparatus 700 of this embodiment may include: a first uploading module 701, a first receiving module 702, a frame selection module 703, a second uploading module 704, and a second receiving module 705. The first uploading module 701 is configured to photograph and upload a multi-question image, wherein the multi-question image includes a plurality of exercises; the first receiving module 702 is configured to receive the coordinates of the plurality of exercises obtained by performing a multi-question detection on the multi-question image; the frame selection module 703 is configured to frame the multi-question image based on the coordinates of the plurality of exercises to obtain a selection frame of the plurality of exercises; the second uploading module 704 is configured to upload a selection frame of at least one exercise among the plurality of exercises; and the second receiving module 705 is configured to receive a search result obtained by searching for an exercise based on at least one exercise.

[0081] In this embodiment, in the photo search device 700, the specific processing of the first upload module 701, the first receiving module 702, the frame selection module 703, the second upload module 704 and the second receiving module 705 and the technical effects thereof can be referred to respectively. Figure 1 The relevant descriptions of steps 101-105 in the corresponding embodiment are not repeated here.

[0082] In some optional implementations of this embodiment, the photo-taking and question-searching device 700 further includes: a marking module configured to mark serial numbers for the boxes of the plurality of exercises.

[0083] In some optional implementations of this embodiment, the second uploading module 704 includes: an uploading submodule configured to upload the selected box in response to selecting a box from a plurality of boxes of exercises.

[0084] In some optional implementations of this embodiment, the second upload module 704 further includes: a presentation submodule configured to present an adjustment button on the selected selection box; and an adjustment submodule configured to adjust the selected selection box in response to detecting an operation of the adjustment button.

[0085] In some optional implementations of this embodiment, the photo-taking question-searching device 700 further includes: a display module configured to display photo-taking guidance information on the photo-taking page.

[0086] In some optional implementations of this embodiment, the display module includes: a first display submodule, configured to display guidance information for installing the image acquisition device and placing the tablet in response to detecting a front shooting scene.

[0087] In some optional implementations of this embodiment, the display module also includes: a second display sub-module, configured to display guidance information for installing the image acquisition device in response to the image acquisition device not being installed; a third display sub-module, configured to display guidance information for placing the tablet in response to the image acquisition device being installed but the tablet angle being incorrect; and a fourth display sub-module, configured to display guidance information for placing the exercise document in response to the image acquisition device being installed and the tablet angle being correct.

[0088] In some optional implementations of this embodiment, the display module includes: a fifth display submodule, configured to display guidance information for placing the tablet and the exercise document in response to detecting the post-shooting scene.

[0089] In some optional implementations of this embodiment, the guidance information is displayed when the photo search function is turned on for the first time, and automatically fades out after being displayed for a preset time.

[0090] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0091] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0092] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0093] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0094] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the photo search method. For example, in some embodiments, the photo search method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the photo search method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the photo search method in any other appropriate manner (e.g., by means of firmware).

[0095] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0097] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0099] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0100] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0101] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0102] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for searching questions by taking photos, comprising: The camera guidance information is displayed on the shooting page. The tablet's hardware gyroscope is used to provide guidance and prompts for various user scenarios. The information is displayed when the camera search function is first turned on and automatically fades out after a preset display time. Taking and uploading a multi-question image, wherein the multi-question image includes a plurality of exercises; receiving coordinates of the plurality of exercises obtained by performing multi-question detection on the multi-question image; Select the multiple-question image based on the coordinates of the multiple exercises to obtain selection frames of the multiple exercises; a checkbox for uploading at least one exercise from the plurality of exercises; receiving a search result obtained by searching for questions based on the at least one question, wherein the server, when detecting the multi-question image, identifies whether the content in the multi-question image is a question, and if it is a question, detects the coordinates of the question, and if it is not a question, continues to process the content, and the processing method includes at least one of the following: coordinate detection, content deletion, content annotation, and content improvement; The displaying of the photo-taking guidance information on the photo-taking page includes: In response to detecting the front shooting scene, displaying guidance information for installing the image acquisition device and placing the tablet; In response to the image acquisition device not being installed, displaying guidance information for installing the image acquisition device; In response to the image acquisition device being installed but the tablet angle being incorrect, displaying guidance information for placing the tablet; In response to the image acquisition device being installed and the flat panel angle being correct, guidance information for placing the exercise document is displayed.

2. The method according to claim 1, wherein: After selecting the multiple-question image based on the coordinates of the multiple exercises to obtain the selection frames of the multiple exercises, the method further includes: The check boxes of the plurality of exercises are marked with serial numbers.

3. The method according to claim 1 or 2, wherein: The selection box for uploading at least one of the plurality of exercises includes: In response to selecting a box from among the boxes of the plurality of exercises, the selected box is uploaded.

4. The method according to claim 3, wherein: Before the upload selection box is selected, it also includes: Presenting adjustment buttons on the selected selection box; In response to detecting that the adjustment button is operated, the selected box is adjusted.

5. The method according to claim 1, wherein: The displaying of the photo-taking guidance information on the photo-taking page includes: In response to detecting the post-shooting scene, guidance information for placing the tablet and the exercise document is displayed.

6. A device for searching questions by taking a photo, comprising: A display module is configured to display photo guide information on the photo page, wherein the hardware gyroscope of the tablet is combined to provide guidance and prompts for various usage scenarios of the user, and is displayed when the photo search function is first turned on, and automatically fades out after a preset display time; A first uploading module is configured to capture and upload a multi-question image, wherein the multi-question image includes a plurality of exercises; A first receiving module is configured to receive the coordinates of the multiple exercises obtained by performing multiple-question detection on the multiple-question image; A frame selection module is configured to perform frame selection on the multiple-question image based on the coordinates of the multiple exercises to obtain a selection frame of the multiple exercises; A second uploading module is configured to upload a selection box of at least one exercise among the plurality of exercises; The second receiving module is configured to receive a search result obtained by searching for a question based on the at least one question, wherein when the server detects the multi-question image, it identifies whether the content in the multi-question image is a question, and if it is a question, it detects the coordinates of the question, and if it is not a question, it continues to process the content, and the processing method includes at least one of the following: coordinate detection, content deletion, content annotation, and content improvement; Wherein, the display module comprises: A first display submodule is configured to display guidance information for installing the image acquisition device and placing the tablet in response to detecting the front shooting scene; A second display submodule is configured to display guidance information for installing the image acquisition device in response to the image acquisition device not being installed; A third display submodule is configured to display guidance information for placing the tablet in response to the image acquisition device being installed but the tablet angle being incorrect; The fourth display submodule is configured to display guidance information for placing exercise documents in response to the image acquisition device being installed and the tablet being at a correct angle.

7. The device according to claim 6, wherein: The device also includes: The marking module is configured to mark the boxes of the plurality of exercises with serial numbers.

8. The device according to claim 6 or 7, wherein: The second uploading module comprises: The uploading submodule is configured to upload the selected box in response to selecting a box from the boxes of the plurality of exercises.

9. The device according to claim 8, wherein: The second uploading module further includes: A presentation submodule, configured to present an adjustment button on the selected selection box; The adjustment submodule is configured to adjust the selected selection box in response to detecting that the adjustment button is operated.

10. The device according to claim 6, wherein: The display module comprises: The fifth display submodule is configured to display guidance information for placing the tablet and the exercise document in response to detecting the post-shooting scene.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images