Processing method, terminal equipment and target equipment

Abstract

The embodiment of the invention provides a processing method, terminal equipment and target equipment. The processing method comprises the following steps: establishing a connection channel with the target equipment; based on the connection channel established with the target device, in response to a target instruction, obtaining a target image including display content of a display screen of the target device; identifying the target image to obtain an identification result; and outputting the identification result. Wherein the target instruction is used for indicating to obtain and recognize the target image and outputting a recognition result of the target image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a processing method, a terminal device, and a target device. Background Art

[0002] Currently, when users use some electronic devices, such as computers, they encounter problems similar to those encountered by users on mobile terminals, such as mobile phones. The problems include:

[0003] When seeing text in a non-native language, I hope to translate the entire interface into my native language;

[0004] When seeing a product image, you want to immediately go to a shopping website and search for the image in order to purchase it;

[0005] When you see an address, you want to immediately find navigation information on how to get to the address.

[0006] When you see a date, you want to create a schedule for that date on your phone.

[0007] When you see a phone call, you want to dial it immediately from your phone, or save it to your phone, etc.

[0008] For the above problems and other problems, users need to go through multiple operations to achieve the desired effect, including selecting a target, exiting the current application, opening the target application, inputting the selected target into the target application to obtain the corresponding result, then exiting the target application, and returning to the original application. When there is no corresponding application locally to process the selected target, the user needs to search for the relevant application online, download it and use it, or use the network to process the selected target. However, no matter which method is used, the overall process is extremely cumbersome and inefficient. Summary of the invention

[0009] In order to solve the above technical problems, the present application provides a processing method, including:

[0010] Establish a connection channel with the target device;

[0011] Based on the connection channel established with the target device, in response to a target instruction, obtaining a target image including display content of a display screen of the target device;

[0012] Recognize the target image to obtain a recognition result;

[0013] Outputting the recognition result;

[0014] The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

[0015] In some embodiments, establishing a connection channel with the target device includes:

[0016] If the target device is within the signal coverage range of the wireless communication module of the terminal device, a connection channel with the target device is established.

[0017] In some embodiments, the responding to the target instruction based on the connection channel established with the target device includes:

[0018] If the connection channel established with the target device is successful, enabling the function of responding to the target instruction;

[0019] If the connection channel established with the target device is unsuccessful, the function of responding to the target instruction is disabled.

[0020] In some embodiments, the enabling of the response to the target instruction comprises:

[0021] If the target instruction is obtained, a target program responds to the target instruction, the target program is related to a target model, and the target model is used to process the target image to generate a recognition result;

[0022] The target program and the target model are related to each other and include the following:

[0023] The target program includes the target model; or;

[0024] The target program can call the target model.

[0025] In some embodiments, if the connection channel established with the target device is successful, at least one target instruction to be enabled is determined based on the descriptive information used to characterize the model processing capabilities of the target device, wherein different target instructions correspond to different local models, the local model is different from the model possessed by the target device, and the target model belongs to the local model.

[0026] In some embodiments, if the connection channel established with the target device is successful, in response to the target instruction, a target image including the display content of the display screen of the target device is identified to obtain a recognition result, and based on the connection channel, a display interface including the recognition result displayed on the local display screen is mirrored and projected to the target device.

[0027] In some embodiments, outputting the recognition result includes:

[0028] The recognition result is outputted at a display content corresponding to the target image on a display screen of the target device.

[0029] In some embodiments, outputting the recognition result includes:

[0030] Based on the connection channel, mirroring the display interface including the recognition result displayed on the local display screen to the target device;

[0031] The method further comprises:

[0032] In a case where the target device obtains and displays the projection image including the recognition result through the connection channel, responding to the obtained new target instruction;

[0033] Recognize the projection image to obtain a new recognition result;

[0034] Outputting the recognition result in a non-projection area of ​​the local display screen;

[0035] Among them, the new target instruction is used to instruct to obtain and identify the projection image and output the recognition result of the projection image.

[0036] Another embodiment of the present application also provides a terminal device, including:

[0037] A first communication connection device, used to establish a connection channel with a target device;

[0038] a first processor, configured to obtain, in response to a target instruction, a target image including display content of a display screen of the target device according to the existence of a connection channel established with the target device, recognize the target image to obtain a recognition result, and output the recognition result;

[0039] The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

[0040] Another embodiment of the present application further provides a target device, including:

[0041] A second communication connection device, used to establish a connection channel with the terminal device;

[0042] The second processor is used to control the acquisition of a target image including the display content of the display screen, generate a target instruction, and send the target instruction and the target image to the terminal device through the connection channel, wherein the target instruction is used to instruct the acquisition and recognition of the target image and output the recognition result of the target image.

[0043] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0044] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a flowchart of the processing method in the embodiment of the present application.

[0047] Figure 2 It is a flowchart of a processing method in another embodiment of the present application.

[0048] Figure 3 This is a structural relationship diagram of the terminal device in the embodiment of the present application.

[0049] Figure 4 2 is a structural relationship diagram of the target device in the embodiment of the present application. DETAILED DESCRIPTION

[0050] Below, specific embodiments of the present application are described in detail with reference to the accompanying drawings, but are not intended to limit the present application.

[0051] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope of the present disclosure will occur to those skilled in the art.

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0053] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0054] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0055] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0056] Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of ways with substantially any suitable detailed structure.

[0057] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0058] Below, embodiments of the present application are described in detail with reference to the accompanying drawings.

[0059] In order to solve the problem that in the existing solution, users need to perform multiple operations to obtain the required processing results, resulting in cumbersome operations and low efficiency, such as Figure 1 As shown, the embodiment of the present application provides a processing method, including:

[0060] S1: Establish a connection channel with the target device;

[0061] S2: based on the existence of the connection channel established with the target device, in response to a target instruction, obtaining a target image including display content of a display screen of the target device;

[0062] S3: Identify the target image to obtain a recognition result;

[0063] S4: outputting the recognition result;

[0064] The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

[0065] The scheme in this embodiment can be, but is not limited to, applied in terminal devices, such as mobile phones, tablet computers, learning machines, etc., without specific limitations. Of course, it can also be a non-terminal device, such as a computer, a server, etc. The target device can be a computer, a learning machine, or a terminal device, etc., and the specific is also not unique. In this embodiment, the method is applied to a terminal device as an example for explanation. The connection channel described in the scheme is a connection channel established between the terminal device and the target device. For example, after the first application in the terminal device is started, a link will be generated with the target device to build a connection channel, or after the first application in the target device is started, a link will be generated with the terminal device to build a connection channel. It can also be that the first application responds to the instructions input by the user, etc., and then generates a link with the terminal device to build a connection channel, etc. The specific way of building a connection channel is uncertain, and the conditions for triggering the construction of the connection channel are also uncertain. For example, it can also be, but not limited to, that the target device responds to the generation of the target instruction and actively establishes a connection channel with the terminal device. After a connection channel is established between the target device and the terminal device, the target device can send the target instruction to the terminal device through the connection channel, or through other connections, such as Bluetooth connection, wireless link, forwarding of the server corresponding to the application currently started in the target device for generating the target instruction, etc. When the terminal device receives the target instruction, it can directly determine that the connection channel exists, that is, it has been established, or when receiving the target instruction, it can determine whether the connection channel has been established by checking the parameters or changes in related parameters generated locally when the connection channel is established. When it is determined that a connection channel has been established between itself and the target device, the terminal device will respond to the target instruction and obtain a target image including the display content of the display screen of the target device. The target image can be part of the content displayed on the display screen of the target device, such as part of the content or all of the content in a certain display frame, or it can be the entire content of the display screen, such as the display screen corresponding to a certain content when the display screen displays a certain content in full screen. The target image can include text, images, etc., which are not unique. The target instruction is used to instruct to obtain and identify the target image and output the recognition result of the target image. Based on this, after obtaining the target image, the terminal device will perform recognition processing on it, that is, recognize the target image and obtain the recognition result. For example, the recognition result may be specific information of an image, including text information in the image, the expression meaning and source of the image. When the image is a product image, the recognition result may also include a purchase link for the product, etc., or the recognition result may be text information, including a translation result in a specified language, etc. It may also be navigation information, created reminder information, contact information of a certain user, etc., which are not specific and unique.After obtaining the recognition result, the terminal device will output the recognition result, such as transmitting the device result to the target device through the connection channel, and outputting the recognition result based on the target device, or the terminal device outputs the recognition result locally, or it can also merge the above two output methods and output in parallel based on different output methods.

[0066] Based on the above method, it can be known that when a user encounters content that he wants to process and obtains the desired result when using the target device, the user himself does not need to operate by himself, call related programs or functions, etc. to process the content to obtain the desired result, but can instruct the target device to establish a connection channel with the terminal device, and then select the content. At this time, the target device can automatically generate a target instruction based on the content and send it to the terminal device. The terminal device executes the target instruction when it determines that the connection channel has been established, recognizes and processes the content, and obtains the corresponding recognition result, and finally outputs the recognition result so that the user can obtain the desired result. The whole process is almost completed by the target device and the terminal device themselves, and does not require frequent user participation. It only needs to determine the content that needs to be processed. Therefore, the user's operation amount is greatly reduced, and the current phenomenon that users need to perform multiple operations by themselves to obtain the desired results is effectively improved. It improves the convenience of users when they want to process specified content on the target device, and at the same time broadens the scope of use of the terminal device and maximizes the resource utilization of the terminal device.

[0067] In one embodiment, establishing a connection channel with the target device includes:

[0068] S5: If the target device is within the signal coverage range of the wireless communication module of the terminal device, a connection channel with the target device is established.

[0069] The conditions for building the connection channel in this embodiment are relatively low, as long as the network requirements are met, specifically, the target device is located within the signal coverage of the wireless communication module of the terminal device, such as the radiation antenna, and the connection channel between the target device and the terminal device can be supported. When the connection channel is built for the first time, a series of security authentications such as address and identity are required. Once the authentication is passed and the connection channel is built between the two, the next time the target device is located within the signal coverage of the terminal device, the target device and the terminal device can automatically establish a connection channel, or they can also ask the user, or when the user enters a gesture, eye contact, touch command, shortcut key, etc. to represent the indication of building a channel, that is, it is established after the user's permission, or when the target device starts a program or function, the target device actively builds a connection channel with the terminal device. Of course, the conditions for triggering the establishment of the channel are not limited to the above examples, and can also be other types of conditions, which are not specific. As long as the construction basis can be met, that is, the target device is located within the coverage of the wireless network signal, Bluetooth signal, etc. of the terminal device, so that there is a network as a basis between the two, and data transmission can be smoothly realized, any additional trigger conditions can be flexibly configured.

[0070] When the terminal device determines whether to execute the target instruction, as described above, it is determined based on whether the connection channel is connected, for example, Figure 2 As shown, the response to the target instruction based on the connection channel established with the target device includes:

[0071] S6: If the connection channel established with the target device is successful, enabling the function of responding to the target instruction;

[0072] S7: If the connection channel established with the target device is unsuccessful, disabling the function of responding to the target instruction.

[0073] If the terminal device determines that the connection channel is successfully established, that is, the network signal coverage of the terminal device stably covers the target device, the connection channel has been established between the two, and the connection channel is stable, then the terminal device determines that it can respond to the target instruction, that is, the function of executing the target instruction is enabled, and the terminal device will perform recognition of the target image to obtain the recognition result. If the terminal device determines that it has not established a connection channel with the target device, or the connection channel is not successfully established, such as the target device does not exist within the network signal coverage of the terminal device, or the target device is unstable within the network signal coverage, so that a stable connection channel cannot be established between the two, the terminal device will disable the ability to respond to the target instruction, that is, the terminal device will not recognize the target image and generate a recognition result, but will discard the target instruction or do nothing about the target instruction.

[0074] In actual application, the terminal device can determine whether the connection channel is established by querying whether there is a target device within the network coverage, or confirming whether a connection request from the target device has been received, or by querying whether there is parameter information of the corresponding connection channel in the established connection.

[0075] Further, the function of enabling response to the target instruction includes:

[0076] S8: If the target instruction is obtained, the target program responds to the target instruction, the target program is related to the target model, and the target model is used to process the target image to generate a recognition result;

[0077] The target program and the target model are related to each other and include the following:

[0078] The target program includes the target model; or;

[0079] The target program can call the target model.

[0080] Exemplarily, if the target instruction is obtained, the terminal device can determine the target program for responding to the target instruction according to a pre-defined protocol, or determine the task that needs to be performed, such as an image recognition task, according to the instruction information in the target instruction, and then determine locally the target program that has the corresponding function or can call the function based on the task so that it can perform the task. The target program is related to the target model. For example, to achieve image recognition, the target model needs to have the ability to perform image recognition and generate recognition results. The relationship between the target program and the target model is not unique. For example, the target program itself has the target model, or the target model is located in the terminal device and the target program has the authority to call the target model, or the target program can call the target model located in the server through the corresponding server, etc.

[0081] After obtaining the recognition result, the terminal device can further determine the user's intention based on the recognition result, and then further process the recognition result based on the user's intention. For example, when it is determined by analyzing the recognition result that the user wants to translate text, obtain navigation information, create a schedule, search for product information contained in the image, send an email to the address in the image, etc., the terminal device can automatically determine the corresponding program to perform the corresponding task, such as instructing the email program to edit the specified content and send it to the specified address, search for the product information and sales link of the illustrated product, translate the non-native language data in the image into the native language, etc., and finally obtain the processing result of the task.

[0082] The content executed by the above-mentioned terminal device can be in a form independent of the execution of the target instruction, that is, the target instruction does not contain the content of executing the task in the image information, and the terminal device executes the task in the image information based on a pre-established protocol or other instructions. Or the content executed by the above-mentioned terminal device is integrated with the execution of the target instruction, that is, the target instruction contains the content that instructs the terminal device to automatically execute the task in the image information. The terminal device does not need to be based on other protocols or instructions, but only needs to execute the content of the target instruction to sequentially execute the image recognition and illustration tasks, and obtain the recognition result corresponding to the result required by the user, that is, the recognition result is actually the execution result of the illustration task, such as the email after sending, the translation result, etc.

[0083] In this embodiment, the target instruction contains information that instructs the terminal device to perform the illustrated task, or hidden information. The target instruction in this embodiment is to instruct the terminal device to recognize the target image, and then automatically trigger the corresponding function based on the recognition result to complete the indication information to be expressed by the target image. For example, if the recognition result of the target image is a certain commodity, the terminal device automatically triggers an e-commerce program to perform a search for the corresponding commodity and obtains a recognition result containing the search result, or if the recognition result of the target image is a paragraph of English, the terminal device automatically triggers a translation program to perform the Chinese translation of the paragraph of English and obtains a recognition result containing the Chinese translation result. Therefore, the target instruction in this embodiment is to instruct the terminal device to perform image recognition, so as to instruct the terminal device to perform different tasks through the image recognition result.

[0084] Furthermore, in this embodiment, when the terminal device determines the specific task to be performed based on the image recognition result, the method further includes:

[0085] S9: If the connection channel established with the target device is successful, determine at least one target instruction to be enabled based on the description information used to characterize the model processing capability of the target device, wherein different target instructions correspond to different local models, the local model is different from the model possessed by the target device, and the target model belongs to the local model.

[0086] For example, the terminal device can interact with the target device through the connection channel or other means to obtain the model and its specific functions locally possessed by the target device, such as a model capable of document translation, a model capable of sending emails, etc. Based on this, the terminal device can obtain the tasks that the target device can perform and the tasks that cannot be performed. When the terminal device obtains the image recognition result based on image recognition, if the recognition result is only document information, such as all English information, Japanese information, etc., then the terminal device can determine that the user wants to translate the data based on this. At this time, the terminal device can call the local model to perform the translation, or it can feed back the data to be translated to the target device to call the translation model of the target device to perform the translation. Or when the terminal device determines that there is no relevant translation model in its own program, and the target device has the required translation model, it can directly feed back the content to be translated to the target device, and call the translation model of the target device to perform the translation.

[0087] Alternatively, when the image recognition result includes multiple different types of instructions, such as tasks such as translation, sending emails, matching associated user identities and contact information, the terminal device will determine which tasks can be executed by the model in the target device and which tasks cannot be executed by the model in the target device, that is, tasks that cannot be executed due to the model not having the corresponding function. Based on the above task division, the terminal device can determine at least one target instruction, that is, determine the target instruction, the task that the target device actually wants the terminal device to execute, or the target instruction contains two parts, the first part is to instruct the terminal device to recognize the target image, and the second part is to instruct the terminal device to determine a specific task instruction based on the recognition result. It can also be understood that the type of the target instruction is to instruct the terminal device to perform image recognition, but the specific instruction content is to instruct the terminal device to determine the task to be executed based on the image recognition result. The task can be, for example, the user identity and contact information shown in the self-matching associated image, so that it can be automatically saved in the address book based on this, which is a task that the target device cannot execute. Based on this judgment result, the terminal device will determine the task information as the target instruction, that is, determine it as the task that the target device actually wants to instruct the terminal device to execute, and then automatically execute the task to generate a recognition result containing the execution result of the task.

[0088] In another embodiment, the method further comprises:

[0089] S10: If the connection channel established with the target device is successful, in response to the target instruction, the target image including the display content of the display screen of the target device is identified to obtain a recognition result, and based on the connection channel, the display interface including the recognition result displayed on the local display screen is mirrored and projected to the target device.

[0090] The scheme shown in this embodiment refers to that after the terminal device determines that the connection channel is successfully established, responds to the target instruction, performs image recognition, determines the specific task, executes the task and obtains the execution result, and generates the recognition result that needs to be presented to the user based on the execution result, that is, when the terminal device is about to output the recognition result, the output method selected by the terminal device is based on the connection channel to feed back the display interface containing the recognition result currently displayed on the local display screen to the display screen of the target device in the form of mirror projection, and displays it through the display screen of the target device. At this time, two completely different window interfaces are displayed on the display screen at the same time, one of the two window interfaces is the original window interface displayed by the target device, and the other is the display interface of the terminal device, or the two window interfaces are exactly the same in typesetting and layout, and only the language type of the text information involved is different. The window interface image fed back by the terminal device can be generated by the terminal device based on the recognition result and the target image processing.

[0091] Alternatively, the terminal device may also output in another manner, for example, outputting the recognition result includes:

[0092] S11: Outputting the recognition result at the display content corresponding to the target image on the display screen of the target device.

[0093] The scheme described in this embodiment refers to the terminal device feeding back the recognition result to the target device through the connection channel, and instructing or controlling the target device to output the recognition result at the display content recorded in the corresponding target image. For example, the target image in this embodiment includes the content in the display interface of the target device. It is assumed that the target image corresponds to the content of a window in the display interface of the target device, that is, the target image generated by the target device by intercepting the display page of the window. The target image may contain information such as English or Japanese, and the user controls the interception of the window content in order to instruct the terminal device to translate it. The terminal device determines the translation task and completes the translation. After obtaining the translation result, it will be fed back to the target device through the connection channel, and instruct the target device to display the translation result at the English or Japanese information corresponding to the window, so as to achieve the effect of Chinese-foreign language comparison and annotation, so as to facilitate the user to better understand the translation result.

[0094] In yet another embodiment, the outputting the recognition result includes:

[0095] S12: mirroring a display interface including the recognition result displayed on a local display screen to the target device based on the connection channel;

[0096] After the mirroring screen is completed, the method further includes:

[0097] S13: When the target device obtains and displays the projection image including the recognition result through the connection channel, responding to the obtained new target instruction;

[0098] S14: Identify the projection image to obtain a new identification result;

[0099] S15: Outputting the recognition result in a non-projection area of ​​the local display screen;

[0100] Among them, the new target instruction is used to instruct to obtain and identify the projection image and output the recognition result of the projection image.

[0101] The solution in this embodiment refers to that after the terminal device determines that the connection channel is established and completes the response to the target instruction, and uses the target device to complete the output of the recognition result in the form of screen projection, the user inputs an instruction based on the projection image displayed by the target device, so that the target device generates a new target instruction and sends it to the terminal device, or the user directly inputs a new target instruction based on the projection image displayed by the terminal device, the terminal device first determines whether the connection channel is successfully established, and whether the projection image has been sent to the target device and displayed by the target device, that is, it is determined whether the projection image containing the recognition result of the previous target instruction has been successfully sent to the target device through the connection channel. If so, the terminal device determines to respond to the new target instruction, identifies it, obtains the recognition result, and then displays the recognition result in the local non-projection area, that is, the original recognition result and the new recognition result do not interfere with each other in display, the recognition result projected to the target device is still displayed normally, and the new recognition result will not be projected to the target device, but displayed in the local non-projection area of ​​the terminal device. For example, when the user does not understand a certain word in the original recognition result, or is unclear about a certain symbol, such as an icon, and sends a new target instruction to the terminal device, the terminal device will perform further recognition processing on the new target instruction based on part of the local projection image when it determines that it can respond to the new target instruction, and obtain the interpretation information of the corresponding word or symbol (i.e., the recognition result), and then display the interpretation information independently in the local non-projection area for the user to view using the terminal device, without affecting the projection image being displayed on the display device.

[0102] like Figure 3 As shown, another embodiment of the present application also provides a terminal device, including:

[0103] A first communication connection device, used to establish a connection channel with a target device;

[0104] a first processor, configured to obtain, in response to a target instruction, a target image including display content of a display screen of the target device according to the existence of a connection channel established with the target device, recognize the target image to obtain a recognition result, and output the recognition result;

[0105] The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

[0106] In one embodiment, establishing a connection channel with the target device includes:

[0107] If the target device is within the signal coverage range of the wireless communication module of the terminal device, a connection channel with the target device is established.

[0108] In one embodiment, the step of responding to a target instruction based on the connection channel established with the target device includes:

[0109] If the connection channel established with the target device is successful, enabling the function of responding to the target instruction;

[0110] If the connection channel established with the target device is unsuccessful, the function of responding to the target instruction is disabled.

[0111] In one embodiment, the enabling of the response to the target instruction includes:

[0112] If the target instruction is obtained, a target program responds to the target instruction, the target program is related to a target model, and the target model is used to process the target image to generate a recognition result;

[0113] The target program and the target model are related to each other and include the following:

[0114] The target program includes the target model; or;

[0115] The target program can call the target model.

[0116] In one embodiment, the first processor is further configured to:

[0117] If the connection channel established with the target device is successful, at least one target instruction to be enabled is determined based on the descriptive information used to characterize the model processing capabilities of the target device, wherein different target instructions correspond to different local models, the local model is different from the model possessed by the target device, and the target model belongs to the local model.

[0118] In one embodiment, the first processor is further configured to:

[0119] If the connection channel established with the target device is successful, in response to the target instruction, the target image including the display content of the display screen of the target device is identified to obtain a recognition result, and based on the connection channel, the display interface including the recognition result displayed on the local display screen is mirrored and projected to the target device.

[0120] In one embodiment, outputting the recognition result includes:

[0121] The recognition result is outputted at a display content corresponding to the target image on a display screen of the target device.

[0122] In one embodiment, the outputting the recognition result includes:

[0123] Based on the connection channel, mirroring the display interface including the recognition result displayed on the local display screen to the target device;

[0124] The first processor is further configured to:

[0125] In a case where the target device obtains and displays the projection image including the recognition result through the connection channel, responding to the obtained new target instruction;

[0126] Recognize the projection image to obtain a new recognition result;

[0127] Outputting the recognition result in a non-projection area of ​​the local display screen;

[0128] Among them, the new target instruction is used to instruct to obtain and identify the projection image and output the recognition result of the projection image.

[0129] like Figure 4 As shown, another embodiment of the present application further provides a target device, including:

[0130] A second communication connection device, used to establish a connection channel with the terminal device;

[0131] The second processor is used to control the acquisition of a target image including the display content of the display screen, generate a target instruction, and send the target instruction and the target image to the terminal device through the connection channel, wherein the target instruction is used to instruct the acquisition and recognition of the target image and output the recognition result of the target image.

[0132] In one embodiment, the second processor is further configured to receive and display the recognition result.

[0133] In one embodiment, the second processor is further configured to generate a new target instruction and send the new target instruction to the terminal device through the connection channel.

[0134] Furthermore, an embodiment of the present application also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the processing method described above is implemented. It should be understood that each solution in this embodiment has the corresponding technical effect in the above method embodiment, which will not be repeated here.

[0135] Furthermore, an embodiment of the present application also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions, which, when executed, cause at least one processor to perform a processing method such as that in the embodiment described above.

[0136] It should be noted that the computer storage medium of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate or transmit a program configured to be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, antenna, optical cable, RF, etc., or any suitable combination of the above.

[0137] In addition, it should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0139] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

Claims

1. A processing method comprising: Establish a connection channel with the target device; Based on the connection channel established with the target device, in response to a target instruction, obtaining a target image including display content of a display screen of the target device; Recognize the target image to obtain a recognition result; Outputting the recognition result; The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

2. The processing method according to claim 1, wherein establishing a connection channel with the target device comprises: If the target device is within the signal coverage range of the wireless communication module of the terminal device, a connection channel with the target device is established.

3. The processing method according to claim 1, wherein the step of responding to a target instruction based on the connection channel established with the target device comprises: If the connection channel established with the target device is successful, enabling the function of responding to the target instruction; If the connection channel established with the target device is unsuccessful, the function of responding to the target instruction is disabled.

4. The processing method according to claim 3, wherein the enabling response to the target instruction comprises: If the target instruction is obtained, a target program responds to the target instruction, the target program is related to a target model, and the target model is used to process the target image to generate a recognition result; The target program and the target model are related to each other and include the following: The target program includes the target model; or; The target program can call the target model.

5. The processing method according to claim 3 or 4, further comprising: If the connection channel established with the target device is successful, at least one target instruction to be enabled is determined based on the descriptive information used to characterize the model processing capabilities of the target device, wherein different target instructions correspond to different local models, the local model is different from the model possessed by the target device, and the target model belongs to the local model.

6. The processing method according to any one of claims 1 to 5, further comprising: If the connection channel established with the target device is successful, in response to the target instruction, the target image including the display content of the display screen of the target device is identified to obtain a recognition result, and based on the connection channel, the display interface including the recognition result displayed on the local display screen is mirrored and projected to the target device.

7. The processing method according to any one of claims 1 to 5, wherein outputting the recognition result comprises: The recognition result is outputted at a display content corresponding to the target image on a display screen of the target device.

8. The processing method according to any one of claims 1 to 5, wherein outputting the recognition result comprises: Based on the connection channel, mirroring the display interface including the recognition result displayed on the local display screen to the target device; The method further comprises: In a case where the target device obtains and displays the projection image including the recognition result through the connection channel, responding to the obtained new target instruction; Recognize the projection image to obtain a new recognition result; Outputting the recognition result in a non-projection area of ​​the local display screen; Among them, the new target instruction is used to instruct to obtain and identify the projection image and output the recognition result of the projection image.

9. A terminal device, comprising: A first communication connection device, used to establish a connection channel with a target device; a first processor, configured to obtain, in response to a target instruction, a target image including display content of a display screen of the target device according to the existence of a connection channel established with the target device, recognize the target image to obtain a recognition result, and output the recognition result; The target instruction is used to instruct obtaining and identifying the target image and outputting the identification result of the target image.

10. A target device, comprising: A second communication connection device, used to establish a connection channel with the terminal device; The second processor is used to control the acquisition of a target image including the display content of the display screen, generate a target instruction, and send the target instruction and the target image to the terminal device through the connection channel, wherein the target instruction is used to instruct the acquisition and recognition of the target image and output the recognition result of the target image.

Images