Content recognition method, device and electronic device

Abstract

The present application discloses a content recognition method, apparatus and electronic device, belonging to the field of communication technologies. The method includes: receiving a first input from a user to the electronic device; in response to the first input, displaying first content in a first area of a first screen of the electronic device, where the first area is within the shooting range of a first camera of the electronic device; and when the first content is acquired by the first camera, recognizing the first content to obtain first information.

Description

Technical Field

[0001] This application belongs to the field of communication technologies, and particularly relates to a content recognition method, apparatus, and electronic device. Background Art

[0002] Currently, with the popularization of the Internet, electronic devices have gradually penetrated into users' daily lives and work. For example, users can use the rear camera of an electronic device to identify some content of interest.

[0003] Generally, when a user is watching a movie using an electronic device, if the user does not recognize the plant that appears in a certain frame of the movie, the user can first trigger the electronic device to capture a screenshot of the frame and store the frame. Then, the user can trigger the electronic device to run a certain image recognition application and upload the frame to the server of the image recognition application for recognition. In this way, the method for an electronic device to recognize content (such as the frame of the above-mentioned movie) is relatively cumbersome. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a content recognition method, apparatus, and electronic device, which can solve the problem that the method for an electronic device to recognize content is relatively cumbersome.

[0005] In a first aspect, the embodiments of this application provide a content recognition method, which includes: receiving a first input from a user to an electronic device; in response to the first input, displaying first content in a first area of a first screen of the electronic device, where the first area is within the shooting range of a first camera of the electronic device; and when first content is obtained through the first camera, recognizing the first content to obtain first information.

[0006] In a second aspect, the embodiments of this application provide a content recognition apparatus, which includes a receiving module, a display module, and a processing module. Among them, the receiving module is configured to receive a first input from a user to an electronic device; the display module is configured to, in response to the first input received by the receiving module, display first content in a first area of a first screen of the electronic device, where the first area is within the shooting range of a first camera of the electronic device; and the processing module is configured to, when first content is obtained through the first camera, recognize the first content to obtain first information.

[0007] In a third aspect, the embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] Fourthly, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] Fifthly, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the method described in the first aspect.

[0010] Sixthly, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, a first input from a user to an electronic device is received; in response to the first input, first content is displayed in a first area of a first screen of the electronic device, and the first area is within the shooting range of a first camera of the electronic device; when the first content is acquired through the first camera, the first content is recognized to obtain first information. Through this solution, after the user triggers the display of content in the shootable area of the first screen of the electronic device by input, the content can be acquired through the camera of the electronic device, and when the content is acquired, the content can be directly recognized to obtain the recognition information corresponding to the content, without the user triggering the electronic device to take a screenshot of the content and trigger the upload of the content to the server of the image recognition application for recognition. In this way, the manner of content recognition by the electronic device is simplified. Description of the Drawings

[0012] Figure 1 A schematic diagram of a content recognition method provided by an embodiment of the present application;

[0013] Figure 2 A schematic diagram of an interface applied to a content recognition method provided by an embodiment of the present application;

[0014] Figure 3 A schematic structural diagram of a content recognition device provided by an embodiment of the present application;

[0015] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0016] Figure 5 A schematic hardware diagram of a server provided by an embodiment of the present application. Detailed Embodiments

[0017] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0018] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the related objects before and after are in an "or" relationship.

[0019] In the related art, when a user is watching a movie using an electronic device, if the user does not recognize the plant that appears in a certain picture of the movie, the user can first trigger the electronic device to take a screenshot of the picture and store the picture; then, the user can trigger the electronic device to run a certain image recognition application and upload the picture to the server of the image recognition application for recognition. In this way, the method for the electronic device to recognize content (such as the picture of the above movie) is relatively cumbersome.

[0020] Furthermore, in the above process, the electronic device needs to save the picture, which will result in the generation of some intermediate files and occupy memory.

[0021] Based on the above problems, the embodiments of the present application provide a content recognition method. After a user triggers the display of a certain content in the shootable area of the first screen of the electronic device through input, the electronic device can directly recognize the content through the camera and obtain the information of the content, without the user triggering the electronic device to take a screenshot of the content and trigger the upload of the content to the server of the image recognition application for recognition. In this way, the method for the electronic device to recognize content is simplified.

[0022] Furthermore, since there is no need to trigger a screenshot of the content, no intermediate files will be generated, thus not affecting the content stored in the local album of the electronic device.

[0023] Next, in conjunction with the accompanying drawings, a detailed description will be given to the content recognition method, device, and electronic device provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0024] As Figure 1As shown in the figure, an embodiment of the present application provides a content recognition method, which includes the following S101 to S103.

[0025] S101. The content recognition device receives a first input from the user to the electronic device.

[0026] Optionally, the above first input may be a touch input, a voice input, or a gesture input of the user. For example, the touch input is a folding input of the user to the first screen of the electronic device. Of course, the first input may also be other possible inputs, and the embodiments of the present application do not limit this.

[0027] S102. In response to the first input, the content recognition device displays first content in a first area of the first screen of the electronic device.

[0028] Among them, the above first area is within the shooting range of the first camera of the electronic device.

[0029] Optionally, the first screen of the above electronic device may be a foldable screen.

[0030] Optionally, the above first camera may be a front camera or a rear camera of the electronic device. Optionally, the electronic device is a foldable screen device, including a first screen and a second screen. The first camera may be a camera on the second screen. After folding the electronic device, the first area of the first screen can be within the shooting range of the second screen camera.

[0031] Optionally, the above first camera may be a rotatable camera.

[0032] Optionally, the above first content may come from the content stored in the electronic device or the screen content of the first screen of the electronic device.

[0033] Exemplarily, the above first content may include any one of the following: web page, image, text, icon, identification code, etc.

[0034] Optionally, the number of the above first content may be one or more.

[0035] Further, in the case where the first content includes multiple contents, the multiple contents may be of the same type or of different types.

[0036] [[ID=3,6]]Exemplarily, assume that the first content includes two contents. In one possible case, both of the two contents are images; in another possible case, one content is an image and the other content is text.

[0037] Optionally, when the electronic device includes a second screen and the first screen includes a first screen area and a second screen area; before S101, the content recognition method provided by the embodiments of the present application may further include: the content recognition device receives an input from the user on the second screen; in response to the input, the screen contents in the first screen area and the second screen area are swapped for display. In this way, the display content on the first screen can be controlled and updated by operating the second screen.

[0038] Optionally, when the first screen includes a first screen area and a second screen area, if the first camera is located in the first screen area, the first area is an area in the second screen area. Thus, when the first content is displayed in the first area, the other areas in the second screen area except the first area can be controlled to be in the screen-off state, or the display color of the other areas in the second screen area except the first area can be updated to black.

[0039] S103. When the first content is acquired by the first camera, the content recognition device recognizes the first content to obtain first information.

[0040] Optionally, when the first input is a folding input, when the electronic device detects that the duration of the first screen of the electronic device in the folded state reaches a preset duration, the recognition mode of the first camera can be started. In this mode, the content recognition device can control the first camera to acquire the first content.

[0041] Optionally, when the first content is acquired by the first camera, content recognition technology can be used to recognize the first content. For example, the first content is recognized through an AI recognition algorithm to obtain the recognition information corresponding to the first content.

[0042] Optionally, when the first content includes multiple contents, the multiple contents can be recognized in sequence according to the arrangement order of the multiple contents; or, the multiple contents can be recognized simultaneously. Specifically, it can be determined according to the actual situation, and the embodiments of the present application do not limit this.

[0043] It should be noted that the first information is determined according to the first content. For different types of first contents, different first information is obtained after the first content is recognized by the first camera.

[0044] Exemplarily, when the first content is an image, the first information is the name of the image; when the first content is an English sentence, the first information is the Chinese translation of the English sentence; when the first content is an icon, the first information is the name of the icon; when the first content is an identity code, the first information is the identity information corresponding to the identity code.

[0045] Exemplarily, take the content recognition device as a folding screen mobile phone and the first camera as the front camera as an example. The user is using the mobile phone to browse an image of Hanfu. If the user does not recognize this Hanfu image, then the user can fold the screen of the mobile phone (i.e., the first screen). After the mobile phone receives the folding input of the screen from the user, in response to this input, the Hanfu image can be displayed in the first area of the screen within the shooting range of the front camera; afterwards, the mobile phone can recognize the Hanfu image through the front camera and obtain the name of the Hanfu, "Tang Dynasty Ruqun" (i.e., the first information).

[0046] Optionally, in the case where the first content includes multiple contents, the multiple contents can be integrated according to the type information of the multiple first information.

[0047] For example, after recognizing multiple texts, the multiple texts can be spliced together for display; for another example, after recognizing multiple texts through the first camera and translating the multiple texts to obtain the translation result, i.e., the first information, then the translation result can be displayed in one-to-one correspondence with the original content, i.e., one line of Chinese and one line of English.

[0048] Optionally, the above first content includes M sub-contents, where M is a positive integer; after the above S101 and before the above S103, the content recognition method provided by the embodiments of the present application may further include: the content recognition device receives a fourth input from the user for the target sub-content among the M sub-contents displayed in the second area. Accordingly, the above S103 may be specifically implemented by the following S103A.

[0049] S103A. The content recognition device, in response to the fourth input, when obtaining the target sub-content through the first camera, recognizes the target sub-content to obtain the first information.

[0050] Optionally, the above fourth input may be a touch input, a voice input, or a gesture input from the user for the target sub-content. For example, the touch input is a click input from the user for the target sub-content.

[0051] Optionally, the number of the above target sub-contents may be one or more.

[0052] Further, the above first information includes M sub-information. When the target sub-content includes one sub-content, the first information includes one sub-information; when the target sub-content includes multiple sub-contents, the first information includes multiple sub-information, and each sub-information corresponds to one sub-content among the multiple sub-contents.

[0053] It should be noted that when recognizing the target sub-content, the first information obtained is the detailed information of the target sub-content.

[0054] Combined with the above exemplary description, the mobile phone displays an image of a Han-style clothing in the first area, and the image of the Han-style clothing includes an image of buttons, an image of patterns, etc. If the user wants to know which type of buttons this Han-style clothing has, the user can click on the image of the buttons. After the mobile phone receives the click input, in response to the click input, the mobile phone can use an AI recognition algorithm to recognize the image of the buttons and obtain the name of the buttons, "pan buttons" (i.e., the first information).

[0055] It can be understood that the user can trigger the recognition of the target sub-content in the first content through the input of the target sub-content in the first content, obtain the first information, and thus can focus on recognizing the specific sub-content in the first content. In this way, the details of the first content can be recognized and the information of the details can be obtained.

[0056] The embodiment of the present application provides a content recognition method. After the user triggers the display of a certain content in the shootable area of the first screen of the electronic device through input, the content can be obtained through the camera of the electronic device, and the content can be directly recognized when the content is obtained to obtain the information of the content, without the user triggering the electronic device to take a screenshot of the content and triggering the upload of the content to the server of the image recognition application. In this way, the way for the electronic device to recognize the content is simplified.

[0057] Optionally, before displaying the first content in the first area of the first screen of the electronic device in S102 above, the content recognition method provided by the embodiment of the present application may further include S104 to S106.

[0058] S104: The content recognition device displays the first content in the second area of the second screen of the electronic device in response to the first input.

[0059] Among them, the above second area is the mapped area of the first area.

[0060] S105: The content recognition device receives the second input of the user on the second area of the second screen.

[0061] S106: The content recognition device updates the first content displayed in the first area in response to the second input.

[0062] Optionally, the electronic device in the embodiment of the present application may be an electronic device with multiple screens. Further, the multiple screens include a first screen and a second screen.

[0063] Optionally, the second screen is located in a plane opposite to the plane where the first screen is located, or the second screen is located in the same plane as the plane where the first screen is located. It is specifically determined according to the actual usage situation, and the embodiment of the present application does not limit this.

[0064] Optionally, the size of the second region and the size of the first region may be the same or different.

[0065] Further, when the size of the second region is different from the size of the first region, the size of the second region is greater than the size of the first region; or, the size of the second region is less than the size of the first region. Specifically, it can be determined according to the actual usage situation, and the embodiments of the present application do not limit this.

[0066] It should be noted that after receiving the first input, the first region can be determined first, and then the mapping region of the first region, that is, the second region, can be determined.

[0067] Further, when the second region is determined, a frame can be displayed on the second screen, and the region enclosed by the frame is the second region. In this way, the user can preview the shootable region of the first camera.

[0068] Optionally, when the first content includes multiple contents, the multiple contents are displayed in the second region of the second screen of the electronic device. After that, if the user drags one content of the multiple contents to another content, then it can be triggered to splice the information obtained after identifying the two contents.

[0069] Optionally, the second input may be any feasible input such as the user's touch input, voice input, or gesture input, and the embodiments of the present application do not make any limitation on this.

[0070] Optionally, the content recognition device receives the second input of the user to the second region, updates the content displayed in the second region, and synchronously updates the first content displayed in the first region. Optionally, the second input may be the user's edit input to the first content in the second region. Exemplarily, the second region of the second screen displays the text A to be recognized, and the user obtains the text B for the text A, and the content recognition device updates the text A in the first region to the text B.

[0071] It should be noted that since there is a mapping relationship between the second region and the first region, after triggering the display of the first content in the second region of the second screen through the first input, the first content in the mapping region of the first region can be simultaneously displayed in the first region of the first screen.

[0072] That is to say, when the content displayed in the second region changes, the content displayed in the first region will also change accordingly.

[0073] The content recognition method provided by the embodiments of the present application can display the first content in the second area of the second screen of the electronic device, and display the first content in the mapping area of the first area in the first area, so that the content displayed in the first area of the first screen can be determined through the content displayed in the second area of the second screen, and the operation is more convenient.

[0074] Optionally, before displaying the first content in the second area of the second screen in S104 above, the content recognition method provided by the embodiments of the present application may further include the following S107 and S108; correspondingly, the above S104 may be specifically implemented by the following S104A.

[0075] S107. The content recognition device displays N contents in the third area of the second screen.

[0076] Among them, the above N contents include the first content, and N is a positive integer.

[0077] Optionally, the above third area is a screen area different from the second area in the second screen.

[0078] Optionally, the sizes of the above second area and the third area may be the same or different.

[0079] Optionally, when the sizes of the second area and the third area are different, the size of the second area is greater than the size of the third area; or, the size of the second area is less than the size of the third area. It can be specifically determined according to the actual usage situation, and the embodiments of the present application do not limit this.

[0080] Optionally, for the description of each of the N contents, reference may be made to the detailed description in the above embodiments, and the embodiments of the present application will not elaborate herein.

[0081] S108. The content recognition device receives a third input from the user for the first content.

[0082] Optionally, the above third input may be a touch input, a voice input or a gesture input from the user for the first content. For example, the touch input is an input in which the user drags the first content from the third area to the second area.

[0083] Optionally, in combination with the above S108, the above step S104 may include the following S104A.

[0084] S104A. The content recognition device displays the first content in the second area in response to the third input.

[0085] Optionally, after the above S108, the content recognition method provided by the embodiments of the present application may further include: The content recognition device responds to a third input and displays an identification of a list to be recognized, where the list to be recognized includes an identification of a list indicating the first content. In this way, the user can view the recognized content and the content to be recognized.

[0086] Exemplarily, take the content recognition device as a mobile phone. As Figure 2 shown, the mobile phone includes a first screen 01 and a second screen 02. The user can fold the first screen 01. After the mobile phone receives the folding input of the user (i.e., the first input), the mobile phone can respond to the folding input and display content 04 and content 05 in area 03 (i.e., the third area) of the second screen 02. Then, the user can drag the content 04 to area 06 (i.e., the second area) of the second screen 02. After the mobile phone receives the dragging input, it can respond to the dragging input and display the content 04 in area 06; furthermore, the content 04 displayed in the mapped area 06 of area 07 can be displayed in area 07 of the first screen.

[0087] In the content recognition method provided by the embodiments of the present application, after N contents are displayed in the third area of the second screen, the user can trigger the display of the first content in the second area by inputting to the N contents, so that the user can select any one of the N contents as the content to be recognized according to actual needs.

[0088] Optionally, before displaying the first content in the first area of the first screen of the electronic device in the above S102, the content recognition method provided by the embodiments of the present application may further include the following S109.

[0089] S109: The content recognition device takes a screenshot of the screen content displayed on the first screen to obtain N contents.

[0090] Optionally, the above S107 specifically includes: The content recognition device takes a screenshot of the screen content displayed on the first screen to obtain at least one image; and processes the at least one image in a preset manner to obtain N contents.

[0091] Optionally, for the above processing of at least one image in a preset manner to obtain N contents, the following two possible implementation manners may be included:

[0092] (1) Extract image elements in at least one image according to the category of the image elements to obtain N contents, that is, the N contents are N images.

[0093] Exemplarily, the image elements in at least one image include people, animals, and plants. According to the category of the image elements, people, animals, and plants can be extracted from the at least one image.

[0094] (2) Divide at least one image according to the interface level to obtain N contents.

[0095] Exemplarily, the at least one image includes 3 interface levels. Divide the at least one image according to the interface level to obtain Content 1 located at the first interface level: pictures and texts; Content 2 located at the second interface level: background; Content 3 located at the third interface level: people.

[0096] It can be understood that by taking a screenshot of the screen content displayed on the first screen, N contents can be obtained, so that the user can select the content to be recognized from the N contents.

[0097] Optionally, the above S109 can be replaced by the following S109A and S109B:

[0098] S109A. The content recognition device performs content parsing on the screen content displayed on the first screen.

[0099] S109B. The content recognition device divides the screen content displayed on the first screen according to the parsing result to obtain N contents.

[0100] Optionally, the content recognition device parses the screen content of the first screen through an AI algorithm to obtain the respective content types of the screen content, and then divides the screen content according to the respective content types to obtain the screen content of each type.

[0101] Exemplarily, when it is recognized that the screen content of the first screen includes two types of content, pictures and texts, the screen content can be divided to obtain the pictures and texts in the screen content respectively (i.e., N contents).

[0102] Optionally, before the above S109 or S109B, the content recognition method provided by the embodiments of the present application may further include the following S110 to S112.

[0103] S110. The content recognition device displays a target window on the second screen.

[0104] Wherein, the content of the target area in the first screen is displayed in the target window; the target area is at least part of the screen area in the first screen except the first area.

[0105] Optionally, the above S110 may specifically include the following two possible implementation manners of (a) and (b):

[0106] (a) The content recognition device superimposes and displays the target window on the second screen.

[0107] (b) The content recognition device floats and displays the target window on the second screen.

[0108] Optionally, the display area of the above-mentioned target window may be a preset area in the second screen; the display size of the target window may be a preset display size; the display shape of the target window may be circular, oval, rectangular or other possible shapes. Specifically, it can be determined according to the actual situation, and the embodiments of the present application do not limit the display form of the target window.

[0109] S111. The content recognition device receives a fourth input from the user to the target window.

[0110] Optionally, the above-mentioned third input may be a touch input, voice input or gesture input of the user to the target window. For example, the touch input is a sliding input of the user on the target window.

[0111] S112. In response to the fourth input, the content recognition device updates the content displayed in the target window, and in accordance with the updated content in the target window, updates the content in the target area of the first screen in real time.

[0112] Exemplarily, in a scenario of continuously recognizing content, such as translating an entire article, a small window is displayed on the second screen, and the small window displays the content of the target area of the first screen in real time. After the user slides the small window, it can trigger an update of the content displayed in the target window, and in accordance with the updated content in the target window, update the content in the target area of the first screen in real time.

[0113] Optionally, in combination with the above S110 to S112, the above S109A may include the following S109A1.

[0114] S109A1. Perform content parsing on the updated content in the first screen.

[0115] Optionally, the above N pieces of content are obtained by dividing the updated content in the first screen.

[0116] Exemplarily, in combination with the above example, when translating an entire article, after the user slides the text to be translated in the small window on the second screen, it triggers an update of the text content displayed in the small window, and synchronously updates the content in the lower half screen area of the first screen, that is, the updated text content in the small window is displayed in the lower half screen area of the first screen. In this case, the content recognition device performs content parsing on the updated text content displayed in the lower half screen area of the first screen to obtain N pieces of content.

[0117] In the content recognition method provided by the embodiment of the present application, since the target window is displayed on the second screen, the user can trigger an update of the content displayed in the target window by inputting to the target window, and update the content in the target area of the first screen in real time according to the updated content in the target window. Thus, a way of interacting between the target window and the first screen is provided.

[0118] Optionally, the above first content includes L second sub - contents, where L is a positive integer;

[0119] Optionally, after the first content is displayed in the first area of the first screen of the electronic device in S102, the content recognition method provided by the embodiment of the present application further includes the following S113.

[0120] S113. The content recognition device receives a sixth input from the user for at least two of the L second sub - contents displayed in the second area.

[0121] Optionally, in combination with the above S113, S103 may include the following steps S103B and S103C.

[0122] S103B. In response to the sixth input, when the content recognition device acquires at least two sub - contents through the first camera, it recognizes the at least two sub - contents to obtain second information corresponding to each of the at least two sub - contents.

[0123] S103C. The content recognition device splices the second information corresponding to each sub - content to obtain the first information.

[0124] Optionally, the above L second sub - contents are obtained by dividing the first content. Exemplarily, if the first content is an article, the second sub - contents are multiple text paragraphs in the article. Another example is that if the first content includes Image 1 with text paragraph A obtained by screenshot and Image 2 with text paragraph B, one of the second sub - contents is Image 1 and the other is Image 2.

[0125] Optionally, the above sixth input can be any feasible input such as the user's touch input, gesture input, or voice input, etc. The embodiment of the present application does not make any limitation on this.

[0126] Optionally, the above sixth input is an input in which the user drags multiple of the above at least two sub - contents to one sub - content.

[0127] Exemplarily, taking the above at least two sub - contents including sub - content 1, sub - content 2, and sub - content 3 as an example. After the user drags sub - content 1 and sub - content 2 to sub - content 3 respectively, the content recognition device splices the recognition information 1 (i.e., the second information) corresponding to sub - content 1, the recognition information 2 corresponding to sub - content 2, and the recognition information 3 corresponding to sub - content 3 to obtain the final recognition information (i.e., the first information).

[0128] For example, taking the first content including image 1 obtained by screenshot and including text paragraph A and image 2 including text paragraph B as an example. After the user drags image 1 displayed on the second screen to image 2, after the content recognition device obtains the image content of image 1 and image 2 through the camera on the first screen, it recognizes the text in image 1 and image 2, respectively obtains the text content in image 1 and the text content in image 2, and then integrates and splices the text content in image 1 and the text content in image 2 to obtain a text content including the text in image 1 and image 2.

[0129] In this way, by integrating and splicing the recognition result information corresponding to multiple sub - contents in the first content, a complete recognition result information corresponding to the first content is obtained, improving the readability of the recognition result information, and thus improving the user experience.

[0130] Optionally, after the above S103 or S103A, the content recognition method provided by the embodiments of the present application may further include the following S114 and S115.

[0131] S114. The content recognition device receives a fifth input from the user to the electronic device.

[0132] Optionally, the above fourth input may be a touch input, a gesture input, or a voice input of the user. For example, the touch input is an input for the user to unfold the first screen of the electronic device.

[0133] S115. The content recognition device displays the first information in response to the fifth input.

[0134] Optionally, for the display of the first information in the above S115, the following two possible implementation manners may be included:

[0135] (1) Display the first information on the first screen of the electronic device.

[0136] (2) Display the first information on the second screen of the electronic device.

[0137] Optionally, after the above S114, the content recognition method provided by the embodiments of the present application may further include: restoring the display of the second content on the first screen, where the second content is the screen content displayed on the first screen before receiving the first input. In this way, the user can continue to view the previous content.

[0138] In the content recognition method provided by the embodiment of the present application, after recognizing the first content through the first camera to obtain the first information, the user can trigger the display of the first information through input, so that the user can view the first information obtained after recognizing the first content.

[0139] In the content recognition method provided by the embodiment of the present application, the execution subject can be a content recognition device. In the embodiment of the present application, taking the content recognition device as an example to execute the content recognition method, the content recognition device provided by the embodiment of the present application is described.

[0140] As Figure 3 shown, the embodiment of the present application provides a content recognition device 200, which may include a receiving module 201, a display module 202, and a processing module 203. Among them, the receiving module 201 is configured to receive a first input from the user to the electronic device; the display module 202 is configured to, in response to the first input received by the receiving module 201, display the first content in a first area of the first screen of the electronic device, and the first area is within the shooting range of the first camera of the electronic device; the processing module 203 is configured to, when the first content is obtained through the first camera, recognize the first content to obtain the first information.

[0141] Optionally, in the embodiment of the present application, the display module is further configured to display the first content in a second area of the second screen of the electronic device before displaying the first content in the first area of the first screen of the electronic device, and the second area is a mapped area of the first area; the receiving module is further configured to receive a second input from the user to the second area of the second screen; the processing module is further configured to update the first content displayed in the first area in response to the second input received by the receiving module.

[0142] Optionally, in the embodiment of the present application, the display module is further configured to display N contents in a third area of the second screen before displaying the first content in the second area of the second screen of the electronic device, where the N contents include the first content, and N is a positive integer; the receiving module is further configured to receive a third input from the user to the first content; the display module is specifically configured to display the first content in the second area in response to the third input received by the receiving module.

[0143] Optionally, in the embodiment of the present application, the processing module is further configured to perform content parsing on the screen content displayed on the first screen; the processing module is further configured to divide the screen content displayed on the first screen according to the parsing result to obtain N contents.

[0144] Optionally, in the embodiments of the present application, the display module is further configured to display a target window on the second screen, and the content of the target area in the first screen is displayed in the target window; the target area is at least a part of the screen area in the first screen except the first area; the receiving module is further configured to receive a fourth input from the user to the target window; the processing module is specifically configured to, in response to the fourth input received by the receiving module, update the content displayed in the target window, and in accordance with the updated content in the target window, update the content in the target area of the first screen in real time; the processing module is specifically configured to perform content parsing on the updated content in the first screen, and the N pieces of content are obtained by dividing the updated content in the first screen.

[0145] Optionally, in the embodiments of the present application, the first content includes M first sub - contents, and M is a positive integer; the receiving module is further configured to receive a fifth input from the user to a target sub - content among the M first sub - contents displayed in the second area; the processing module is specifically configured to, in response to the fifth input received by the receiving module, identify the target sub - content when the target sub - content is acquired by the first camera, and obtain first information.

[0146] Optionally, in the embodiments of the present application, the first content includes L second sub - contents, and L is a positive integer; the receiving module is further configured to receive a sixth input from the user to at least two sub - contents among the L second sub - contents displayed in the second area; the processing module is specifically configured to, in response to the sixth input received by the receiving module, identify the at least two sub - contents when the at least two sub - contents are acquired by the first camera, and obtain second information corresponding to each of the at least two sub - contents; the processing module is specifically configured to splice the second information corresponding to each sub - content to obtain first information.

[0147] In the content recognition device provided in the embodiments of the present application, the content recognition device receives a first input from the user to the electronic device; in response to the first input, a first content is displayed in a first area of the first screen of the electronic device, and the first area is within the shooting range of the first camera of the electronic device; when the first content is acquired by the first camera, the first content is recognized to obtain first information. Through this solution, when a certain content is acquired in the shootable area of the first screen of the electronic device by the first camera, the acquired content is triggered to be recognized, and the recognition information corresponding to the content is obtained, without the user first triggering a screenshot of the content and triggering the upload of the content to the server of the image recognition application for recognition. In this way, the way for the electronic device to recognize content is simplified.

[0148] The content recognition device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0149] The content recognition device in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0150] The content recognition device provided by the embodiments of the present application can implement Figure 1 and Figure 2 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0151] Optionally, as Figure 4 shown, the embodiments of the present application further provide an electronic device 300, including a processor 301 and a memory 302. A program or instruction that can run on the processor 301 is stored on the memory 302. When the program or instruction is executed by the processor 301, it implements each step of the above content recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0152] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0153] Figure 5 It is a schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0154] The electronic device 400 includes, but is not limited to, components such as a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, and a processor 410.

[0155] Those skilled in the art can understand that the electronic device 400 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 410 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 5 The structure of the electronic device shown does not limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0156] Among them, the user input unit 407 is used to receive a first input from the user to the electronic device; the display unit 406 is used to display first content in a first area of the first screen of the electronic device in response to the first input received by the user input unit 407, and the first area is within the shooting range of the first camera of the electronic device; the processor 410 is used to identify the first content when the first content is obtained through the first camera, so as to obtain first information.

[0157] Optionally, in the embodiments of the present application, the display unit 406 is further used to display the first content in a second area of the second screen of the electronic device before displaying the first content in the first area of the first screen of the electronic device, and the second area is a mapped area of the first area; the user input unit 407 is further used to receive a second input from the user to the second area of the second screen; the processor 410 is further used to update the first content displayed in the first area in response to the second input received by the user input unit 407.

[0158] Optionally, in the embodiments of the present application, the display unit 406 is further used to display N pieces of content in a third area of the second screen before displaying the first content in the second area of the second screen of the electronic device, where the N pieces of content include the first content and N is a positive integer; the user input unit 407 is further used to receive a third input from the user to the first content; the display unit 406 is specifically used to display the first content in the second area in response to the third input received by the user input unit 407.

[0159] Optionally, in the embodiments of the present application, the processor 410 is further used to perform content parsing on the screen content displayed on the first screen; the processor 410 is further used to divide the screen content displayed on the first screen according to the parsing result to obtain N pieces of content.

[0160] Optionally, in the embodiment of the present application, the display unit 406 is further configured to display a target window on the second screen, where the content of the target area on the first screen is displayed in the target window; the target area is at least a part of the screen area on the first screen except the first area; the user input unit 407 is further configured to receive a fourth input from the user to the target window; the processor 410 is specifically configured to, in response to the fourth input received by the user input unit 407, update the content displayed in the target window, and in accordance with the updated content in the target window, update the content in the target area of the first screen in real time; the processor 410 is specifically configured to perform content parsing on the updated content on the first screen, and the N pieces of content are obtained by dividing the updated content on the first screen.

[0161] Optionally, in the embodiment of the present application, the first content includes M first sub - contents, where M is a positive integer; the user input unit 407 is further configured to receive a fifth input from the user to the target sub - content among the M first sub - contents displayed in the second area; the processor 410 is specifically configured to, in response to the fifth input received by the user input unit 407, when the target sub - content is acquired through the first camera, identify the target sub - content to obtain the first information.

[0162] Optionally, in the embodiment of the present application, the first content includes L second sub - contents, where L is a positive integer; the user input unit 407 is further configured to receive a sixth input from the user to at least two sub - contents among the L second sub - contents displayed in the second area; the processor 410 is specifically configured to, in response to the sixth input received by the user input unit 407, when at least two sub - contents are acquired through the first camera, identify the at least two sub - contents to obtain the second information corresponding to each of the at least two sub - contents; the processor 410 is specifically configured to splice the second information corresponding to each sub - content to obtain the first information.

[0163] In the electronic device provided in the embodiment of the present application, the electronic device receives a first input from the user; in response to the first input, the first content is displayed in the first area of the first screen of the electronic device, and the first area is within the shooting range of the first camera of the electronic device; when the first content is acquired through the first camera, the first content is identified to obtain the first information. Through this solution, when a certain content is acquired in the shootable area of the first screen of the electronic device through the first camera, the acquired content is triggered to be recognized, and the recognition information corresponding to the content is obtained, without the user first triggering a screenshot of the content and triggering the upload of the content to the server of the image recognition application for recognition. In this way, the way for the electronic device to recognize content is simplified.

[0164] It should be understood that in the embodiments of the present application, the input unit 404 may include a graphics processing unit (GPU) 4041 and a microphone 4042. The graphics processing unit 4041 processes the image data of static pictures or videos obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also referred to as a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. The other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0165] The memory 409 can be used to store software programs and various data. The memory 409 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 409 can include a volatile memory or a non-volatile memory, or the memory 409 can include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a direct rambus RAM (DRRAM). The memory 109 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0166] The processor 410 may include one or more processing units; optionally, the processor 410 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor may not be integrated into the processor 410 either.

[0167] The embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned content recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0168] Among them, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disc, etc.

[0169] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the above-mentioned content recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0170] It should be understood that the chip mentioned in the embodiment of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0171] The embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above-mentioned content recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0172] It should be noted that in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in a different order from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0173] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0174] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A content recognition method, characterized in that, The method includes: Receiving a first input from a user to an electronic device; In response to the first input, displaying first content in a second area of a second screen of the electronic device and a first area of a first screen of the electronic device, where the first area is within the shooting range of a first camera of the electronic device, and there is a mapping relationship between the second area and the first area; When the first content is acquired through the first camera, identifying the first content to obtain first information; In response to an edit input to the first content displayed in the second area, updating the first content displayed in the second area and the first area.

2. The method according to claim 1, wherein Before displaying the first content in the first area of the first screen of the electronic device, the method further includes: Displaying the first content in a second area of a second screen of the electronic device; Receiving a second input from the user to the second area of the second screen; In response to the second input, updating the first content displayed in the first area.

3. The method according to claim 2, wherein Before displaying the first content in the second area of the second screen of the electronic device, the method further includes: Displaying N contents in a third area of the second screen, where the N contents include the first content, and N is a positive integer; Receiving a third input from the user to the first content; The displaying the first content in the second area of the second screen of the electronic device includes: In response to the third input, displaying the first content in the second area.

4. The method according to claim 1, wherein Before displaying the first content in the first area of the first screen of the electronic device, the method further includes: Performing content parsing on the screen content displayed on the first screen; Dividing the screen content displayed on the first screen according to the parsing result to obtain N contents.

5. The method according to claim 4, characterized in that, Before dividing the screen content displayed on the first screen according to the parsing result to obtain N contents, the method further includes: Displaying a target window on the second screen, where the content of a target area on the first screen is displayed in the target window; the target area is at least part of the screen area on the first screen other than the first area; Receiving a fourth input from the user to the target window; In response to the fourth input, updating the content displayed in the target window, and in real time updating the content in the target area of the first screen according to the updated content in the target window; The performing content parsing on the screen content displayed on the first screen includes: Performing content parsing on the updated content on the first screen, and the N contents are obtained by dividing the updated content on the first screen.

6. The method according to claim 2, wherein The first content includes M first sub - contents, and M is a positive integer; After displaying the first content in the first area of the first screen of the electronic device, the method further includes: Receiving a fifth input from the user to a target sub - content among the M first sub - contents displayed in the second area; When the first content is acquired through the first camera, identifying the first content to obtain first information, including: In response to the fifth input, when the target sub-content is acquired by the first camera, the target sub-content is identified to obtain first information.

7. The method according to claim 2, characterized in that, The first content includes L second sub-contents, where L is a positive integer; After the first content is displayed in the first area of the first screen of the electronic device, the method further includes: Receiving a sixth input from the user for at least two of the L second sub-contents displayed in the second area; When the first content is acquired by the first camera, identifying the first content to obtain first information, including: In response to the sixth input, when the at least two sub-contents are acquired by the first camera, identifying the at least two sub-contents to obtain second information corresponding to each of the at least two sub-contents; Concatenating the second information corresponding to each sub-content to obtain first information.

8. A content recognition device, characterized in that, The apparatus includes: a receiving module, a display module, and a processing module, where: The receiving module is configured to receive a first input from the user to the electronic device; The display module is configured to, in response to the first input received by the receiving module, display the first content in the second area of the second screen and the first area of the first screen of the electronic device, where the first area is within the shooting range of the first camera of the electronic device, and the second area has a mapping relationship with the first area; The processing module is configured to, when the first content is acquired by the first camera, identify the first content to obtain first information; The processing module is further configured to, in response to an edit input to the first content displayed in the second area, update the first content displayed in the second area and the first area.

9. An electronic device, characterized in that, It includes a processor and a memory, and the memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the content recognition method according to any one of claims 1-7 are implemented.

10. A readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium. When the program or instruction is executed by the processor, the steps of the content recognition method according to any one of claims 1-7 are implemented.

Images