Information processing method and first electronic device

Abstract

The application discloses an information processing method and a first electronic device. The information processing method is applied to the first electronic device and includes the following steps: obtaining an input signal in a process of outputting image data; the image data is obtained based on a screen projection channel between the first electronic device and a second electronic device; in response to being in a first display mode, responding to the input signal based on a first processing mechanism; in response to being in a second display mode, responding to the input signal based on a second processing mechanism; and the first processing mechanism and the second processing mechanism are different in the electronic device for generating an output signal corresponding to the input signal.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to an information processing method and a first electronic device. Background Technology

[0002] With the widespread adoption of smart devices and the development of technology, screen projection has become an important way to share content between multiple devices. However, due to the diversity of projected content and the unpredictability of user actions, accidental touches can easily occur during projection. For example, a user might accidentally touch the large screen during a presentation, causing the displayed PowerPoint slides to flip directly. Summary of the Invention

[0003] The purpose of this application is to provide an information processing method and a first electronic device.

[0004] In a first aspect, embodiments of this application provide an information processing method applied to a first electronic device, comprising: During the output process based on image data, an input signal is obtained; the image data is obtained based on a projection channel between the device and the second electronic device. In response to being in a first display mode, the input signal is responded to based on a first processing mechanism; In response to being in a second display mode, the input signal is responded to based on a second processing mechanism; The electronic devices that generate the output signal corresponding to the input signal under the first processing mechanism and the second processing mechanism are different.

[0005] In one possible implementation, responding to the input signal based on a first processing mechanism includes: Generate an output signal based on the input signal; The output signal is displayed accordingly.

[0006] In one possible implementation, generating an output signal based on an input signal and displaying the output signal includes at least one of the following: The input signal is determined to represent the movement of the operator's finger, a first output signal is generated, and additional content is displayed based on the first output signal, wherein the layer of the additional content is above the layer of the image data; The input signal is determined to represent the scaling of the operator's finger, a second output signal is generated, and the display size of the image data is adjusted based on the second output signal.

[0007] In one possible implementation, the information processing method further includes at least one of the following: In response to being in a first display mode, the operation type corresponding to the input signal is not responded to; the operation type includes one or more of click, touch of the entire screen area, and touch of a portion of the screen area. In response to being in the first display mode, the third electronic device is enabled, a connection is established with the third electronic device, and a prompt is displayed indicating that the operator in the current scene can use the third electronic device to control the display of the first electronic device; In response to being in the first display mode, pop-up messages in the image data are blocked; In response to being in the first display mode, the first model is invoked to perform the generation task.

[0008] In one possible implementation, responding to the input signal based on a second processing mechanism includes: Determine the type of the input signal; In response to the input signal being of a first type, the first input signal is transmitted to the second electronic device so that the second electronic device generates an output signal; In response to the input signal being of the second type, a second input signal is determined based on the first input signal, and the second input signal is transmitted to the second electronic device so that the second electronic device generates an output signal; the data formats of the first input signal and the second input signal are different.

[0009] In one possible implementation, in response to the input signal being of the second type, determining the second input signal based on the first input signal includes: The input signal is analyzed to obtain key value data, and the key value data is converted into continuous coordinate data; The second input signal is determined based on the continuous coordinate data; the operation corresponding to the second input signal is the same as that corresponding to the first input signal.

[0010] In one possible implementation, the information processing method further includes: Image data is input into a second model to obtain the display mode output by the second model; the second model can extract multi-dimensional features of the image data to determine the corresponding display mode; wherein, the second model adopts multi-channel input, and the multi-dimensional features include features of different regions in the image.

[0011] In one possible implementation, multidimensional features of the image data are extracted to determine the corresponding display mode, including... Extract features from the first region of the image to identify the title bar; Extract features from the second region of the image and recognize the taskbar; Extract features from the third region of the image to identify the navigation directory; Extract features from the fourth region of the image to identify the remarks column; The display mode is determined based on the recognition results.

[0012] Secondly, embodiments of this application also provide a first electronic device, comprising: A communication interface is used to establish a projection channel with a second electronic device; A display screen is used to display image data transmitted through a projection channel; Memory, used to store computer instructions; Processor, for loading the computer instructions and executing at least the following: During the output process based on image data, an input signal is obtained; in response to being in a first display mode, the input signal is responded to based on a first processing mechanism; in response to being in a second display mode, the input signal is responded to based on a second processing mechanism; the electronic devices that generate the output signal corresponding to the input signal under the first processing mechanism and the second processing mechanism are different.

[0013] In one possible implementation, The processor is also used to run a second model, which extracts multidimensional features from the image data to determine a corresponding display mode. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of an information processing method provided in this application is shown; Figure 2 A schematic diagram of the model structure of the second model provided in this application is shown; Figure 3 A schematic diagram of the structure of a first electronic device provided in this application is shown. Detailed Implementation

[0016] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0017] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0018] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0019] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0020] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0021] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0022] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0023] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0024] To facilitate understanding of this application, a detailed description of an information processing method provided in this application will be given first. Here, the information processing method of this application embodiment is applied in a scenario where a second electronic device and a first electronic device establish a communication connection, and a projection channel is established between the second and first electronic devices. The second electronic device can then transmit image data to the first electronic device through the projection channel, enabling the first electronic device to display the image data. As an example, the second electronic device can be a terminal device such as a laptop, tablet, or mobile phone, and the first electronic device can be an extended screen, projection device, etc. Typically, the display area of ​​the first electronic device is larger than the display area of ​​the second electronic device.

[0025] As an example, Figure 1 A flowchart of the information processing method provided in an embodiment of this application is shown, wherein, Figure 1The steps shown are executed by the first electronic device, and the specific steps include S101-S103.

[0026] S101, during the output process based on image data, an input signal is obtained; the image data is obtained based on the projection channel between the second electronic device and the projection channel.

[0027] S102, in response to being in the first display mode, responds to the input signal based on the first processing mechanism.

[0028] S103, in response to being in the second display mode, responds to the input signal based on the second processing mechanism.

[0029] The embodiments of this application are described using the example of an operator displaying presentation content through a first electronic device. A projection channel is established between the first and second electronic devices, and the first electronic device obtains image data based on this projection channel. For example, the operator controls the second electronic device to transmit image data to the first electronic device. The image data can be one or more of text, images, audio, and video.

[0030] During the operator's presentation, i.e., during the output process of the first electronic device based on the acquired image data, an input signal is obtained. This input signal is used to control the display of the first electronic device, such as the displayed content and display area. In this embodiment, the input signal can be generated by the operator through the first electronic device or by a second electronic device. The input signal can be a gesture signal, a voice signal, or an electrical signal (such as signals generated by a keyboard, mouse, or touchpad).

[0031] For example, when the first electronic device outputs image data, it can do so through a first display mode or a second display mode. In this embodiment, the display mode does not require the device to have two specific modes; it can be understood as the rendering result based on the display method of the projected data. Different display methods result in different display modes. This embodiment focuses on the different logic used to respond to user input when the projected content is presented in different display methods. For example, in a scenario of projecting a PPT, if the presentation is in slideshow view, the first electronic device is considered to be outputting through the first display mode; if the presentation is in normal view, the first electronic device is considered to be outputting through the second display mode. It should be noted that the above two display models are only two example modes in the PPT projection scenario; other scenarios that can be used in this embodiment are also within the scope of protection of this solution.

[0032] The processing mechanism for responding to input signals differs depending on the display mode of the first electronic device. In this embodiment, the input signal is responded to based on a first processing mechanism in response to a first display mode; and based on a second processing mechanism in response to a second display mode. The electronic devices that generate the output signal corresponding to the input signal differ between the first and second processing mechanisms.

[0033] This application sets up a corresponding processing mechanism to respond to input signals based on the display mode of the first electronic device, thereby avoiding display errors caused by the operator accidentally touching the first or second electronic device and greatly improving the operator's experience.

[0034] Optionally, the first electronic device is equipped with a processing chip or other processing unit to perform functions such as signal response, data processing, and signal output. Based on this, when the first electronic device responds to an input signal according to the first processing mechanism, its processing unit can respond to the input signal to generate an output signal based on the input signal, and then display the signal based on the output signal. Generating an output signal based on an input signal and displaying the signal based on the output signal includes at least one of the following: The first type The first electronic device is equipped with an image acquisition device, such as a camera. In the first display mode, the first electronic device can acquire image information within its acquisition range through the image acquisition device, and analyze and process it to determine whether an input signal exists in the image information, and if so, what the input signal represents. The input signal may correspond to the operator's limbs and changes, etc.

[0035] When the first electronic device acquires image information and determines an input signal representing the movement of the operator's finger based on the image information, it generates a first output signal. The first output signal is used to instruct the generation and display of additional content, which is determined based on the trajectory formed by the movement of the operator's finger.

[0036] After the first output signal is generated, supplementary content is displayed based on the first output signal. This supplementary content can include underlines, annotations, arrows, markers, etc., to assist the presentation.

[0037] Optionally, the additional content layer is above the image data layer, and if there are multiple additional contents, they can be located on the same layer, or they can be located on different layers. That is, the additional content is independent of the image data, and the additional content can be deleted, for example, by using a corresponding erase command, or by deleting the additional content when switching the displayed image data.

[0038] The second type Similarly, when the first electronic device obtains image information through the image acquisition device and determines the input signal representing the scaling of the operator's finger based on the image information, it generates a second output signal, which is used to indicate whether the current display size is reduced or increased.

[0039] After generating the second output signal, the display size of the image data is adjusted based on the second output signal, such as adjusting the size of the current display interface according to the scaling ratio of the finger.

[0040] It is worth noting that when the display screen of the first electronic device has a touch function, the operator can also perform input operations based on the display screen of the first electronic device, and the first electronic device can also obtain the operator's input signal based on the display screen. For example, the coordinates of the operator's finger on the display screen are obtained to form a movement trajectory, the meaning represented by the input signal is determined based on the movement trajectory, and then an output signal is generated based on the input signal, and then the display is performed based on the output signal.

[0041] When the first electronic device is in the first display mode, it can autonomously obtain and respond to input signals without interacting with the second electronic device, which improves the response speed to a certain extent and reduces the consumption of interaction resources.

[0042] In this embodiment of the application, when the first electronic device is in the first display mode, in order to ensure that the display of the first electronic device is not affected, at least one of the following response mechanisms may be included: First response mechanism When the display screen of the first electronic device has a touch function, in response to being in the first display mode, it does not respond to the operation type corresponding to the input signal, thereby effectively preventing the operator from accidentally touching and avoiding misoperation of the first electronic device and / or the second electronic device.

[0043] The operation types include one or more of the following: tapping the entire screen area, tapping or double-tapping a portion of the screen area. For example, disabling touch functionality for the entire screen area, or disabling touch functionality for a portion of the screen area (such as disabling a portion of the screen area excluding the toolbar area).

[0044] The second response mechanism When the first electronic device is capable of communicating with other electronic devices besides the second electronic device, a third electronic device is enabled in response to being in a first display mode. Optionally, the third electronic device can be a laser pointer, signal transmitter, etc. Subsequently, a connection is established with the third electronic device and a prompt is displayed indicating that the operator in the current scenario can use the third electronic device to control the display of the first electronic device, thereby improving the convenience, efficiency, and richness of control during the operator's presentation.

[0045] The third response mechanism During the operator's presentation, to prevent the content displayed by the first electronic device from being obscured, the first electronic device can identify and analyze the image data transmitted by the second electronic device, thereby filtering the image data and allowing the first electronic device to display only a portion of the image data. Based on this, in response to being in the first display mode, pop-up messages in the image data are blocked.

[0046] For example, when the first electronic device is in the first display mode, if there are pop-up messages in the image data transmitted by the second electronic device, such as chat pop-ups in social software or system operation status prompts, the pop-up message will be blocked, that is, the first electronic device will not display the pop-up message, preventing the speech content from being covered and effectively ensuring a better speech effect.

[0047] The fourth response mechanism Considering that the scenarios in which operators give presentations are typically meetings or lectures, there is a need to record the presentation content. Therefore, in response to being in the first display mode, the first model is invoked to execute the generation task. The first model is pre-trained and set up on the first electronic device. The generation task includes recording the presentation content and generating a summary. Correspondingly, the first electronic device automatically starts the screen recording function to execute the generation task.

[0048] Of course, when the first electronic device is in the first display mode, it can also trigger some functions in response to being in the first display mode, such as optimizing the display effect to improve the viewer's viewing experience, discarding handshake signals and query signals sent by unauthenticated electronic devices, etc., to a large extent avoiding the display content being affected, thereby ensuring the presentation effect of the speech and the viewing experience.

[0049] In this embodiment, the first electronic device and the second electronic device each include different input protocols. For example, the first electronic device can respond to touch signals, voice signals, and gesture signals, that is, it supports protocols such as USB Touch (touch input protocol), USB Audio Class (voice input protocol), I²S (voice input protocol), and PCM (voice input protocol); the second electronic device can respond to keyboard signals and mouse signals, that is, it supports protocols such as USB HID (wired keyboard / mouse input protocol) and 2.4G proprietary wireless protocol (wireless keyboard / mouse input protocol). Based on this, when the first electronic device is in the second display mode and responds to the input signal according to the second processing mechanism, the type of input signal is first determined. In this embodiment, the input signal is divided into two types: a first type and a second type.

[0050] For example, in response to an input signal of type 1, the first input signal is transmitted to a second electronic device to cause the second electronic device to generate an output signal. That is, the first type of input signal can be recognized and processed by the second electronic device, meaning that the second electronic device supports the protocol corresponding to the first type of input signal.

[0051] In another example, in response to an input signal being of the second type, a second input signal is determined based on the first input signal, and the second input signal is transmitted to a second electronic device to cause the second electronic device to generate an output signal. Here, the second type of input signal cannot be recognized and processed by the second electronic device; that is, the second electronic device does not support the protocol corresponding to the second type of input signal.

[0052] The first input signal and the second input signal have different data formats. Therefore, when determining the second input signal based on the first input signal in response to the input signal being of the second type, the input signal is parsed to obtain key-value data, which is then converted into continuous coordinate data. The second input signal is determined based on continuous coordinate data; the operation corresponding to the second input signal is the same as that of the first input signal, that is, the output signal generated in response to the first input signal is the same as the output signal generated in response to the second input signal.

[0053] In this embodiment, in addition to the display mode, the factors that trigger the response to the input signal based on the second processing mechanism also need to consider whether the input signal is supported by the protocol of the second electronic device. Furthermore, to improve accuracy, the second mechanism is not triggered under all circumstances; the user needs to perform some operation, such as sliding a finger on the touchscreen. Finger sliding detection is also part of type recognition. Taking the interaction of sliding a finger on the first device to trigger the second electronic device to turn pages in a PPT as an example, in order to achieve this function, this embodiment needs to store the pixel coordinates generated by each finger touch in a queue. If the vertical Y-axis difference of the touch point pixel does not exceed 50px and the horizontal X-axis sliding speed is greater than 800px / s, it will be determined as a sliding gesture to turn pages; otherwise, it will be determined as the user wanting to draw lines or add annotations (i.e., finger movement).

[0054] For example, in this embodiment of the application, a second model is pre-trained to determine the display mode of the first electronic device. This second model is also set in the first electronic device; after obtaining image data, the image data is input into the second model to obtain the display mode output by the second model.

[0055] Optionally, the second model in this embodiment can extract multi-dimensional features of image data to determine the corresponding display mode. That is, the second model performs identification and analysis on the input image data, obtains the analysis result (i.e., the display mode), and outputs it. The second model uses multi-channel input, and the multi-dimensional features include features from different regions of the image.

[0056] For example, Figure 2 A schematic diagram of the model structure of the second model is shown, for reference. Figure 2 It can be seen that the overall structure of the second model can be set as follows: specify the input size, such as the image size corresponding to the input image data being 512. 512 15. The image size is obtained by stitching together five RGB region images. When extracting features from the image data, a combination of the YOLO11s convolutional backbone network and the C3k2 module is used to achieve a balance between model lightweighting and feature extraction capabilities. This allows the model to run on embedded systems or actual terminal devices, making it widely applicable and enabling high-performance recognition in various scenarios. Simultaneously, a combination of a squeeze-and-excitation (SEAttention) module, a convolutional block attention module (CBAM), and a C2f module with pointwise spatial attention (C2fwith PSA, C2PSA) is used to form a relatively stable attention module. This combines channel attention and spatial attention, effectively enhancing the perception of key regions corresponding to the image data (such as title areas, content areas, page numbers, etc.) and improving the recognition accuracy of fine-grained structures. A binary classification system is used to determine the display mode and output based on the extracted features.

[0057] For example, corresponding to the specified input size mentioned above, the second model employs a multi-channel input design. After the image data is input into the second model, it crops the original image corresponding to the image data to form the following structural region: Top: The presence of a title bar in the original image can be identified through this Top area; Bottom: This bottom area can be used to identify whether the taskbar has disappeared in the original image; Left (left side): The navigation directory in the original image can be identified through this Left region; Right: The annotation information bar in the original image can be identified through this Right area; Full: Provides a complete reference image.

[0058] in, H represents the height of the original image, W represents the width of the original image, h = H / 5 represents the height of the structural region, and w = W / 5 represents the width of the structural region.

[0059] The five diagrams corresponding to the five structural regions mentioned above have a size of 512. 512, which is then concatenated into a 15-channel input (3 channels) (5 images) to form the output size requirements of the second model mentioned above.

[0060] Correspondingly, when extracting multidimensional features from image data to determine the corresponding display mode, features of the first region (Top region), the second region (Bottom region), the third region (Left region), and the fourth region (Right region) of the image are extracted respectively. The title bar is identified based on the features of the first region, the task bar based on the features of the second region, the navigation menu based on the features of the third region, and the remarks bar based on the features of the fourth region. Then, the display mode is determined based on the recognition results. This embodiment of the application identifies and analyzes visual features such as the layout, regional structure, and text arrangement of image data, logically divides the original image into regions, and identifies different regions separately, that is, extracts semantic and structural information. This avoids misjudgment caused by complex or unstructured background content during screen projection, effectively improving the accuracy of recognition, and thus improving the accuracy of the output results, thereby ensuring the accuracy of the determined display mode.

[0061] For example, the enhancement mechanism of the attention module in the second model can be referred to as follows: 1) Use the CBAM (Channel + Spatial Attention) module to enhance the model's ability to extract features from key regions of the original image.

[0062] The global average pooling in the attention module refers to the following formula (1): (1) in, This indicates that the c-th channel is globally averaged pooled. This represents the pixel at coordinate (i, j) of the c-th channel of the feature map.

[0063] For example, the channel weights are generated using the following formula (2): (2) in, Indicates channel weight, Indicates the weights of the fully connected layer. The activation function is represented as follows: , This indicates the channel pooling characteristic.

[0064] After determining the channel weights, the weighted channel output is performed according to the following formula (3): (3) in, This represents the weighted feature of the c-th channel. Represents the original c-th channel feature. This represents the weight of the c-th channel.

[0065] (ii) Use the SEAttention module to optimize channel response and improve the accuracy of task-related region recognition.

[0066] The channel pooling operation is performed according to the following formulas (4) and (5): (4) (5) in, This indicates average pooling of the c-th channel. This indicates max pooling of the c-th channel. This represents the original c-th channel feature.

[0067] For example, the spatial attention weights are generated using the following formula (6): (6) in, Represents spatial attention weights. This represents the convolution operation. This indicates a splicing operation.

[0068] The weighted feature map is output using the following formula (7): (7) in, This represents a weighted feature map. This represents the spatial attention weights.

[0069] (iii) Use the C2PSA module to fuse features from multiple regions to improve spatial sensitivity.

[0070] The attention mechanism operation during region fusion can be referenced by the following formula (8): (8) in, Indicates the characteristics after fusion. Indicates the characteristics of the Top region. Indicates the characteristics of the Bottom region. Indicates the characteristics of the Left region. Indicates the characteristics of the Right region. Indicates the characteristics of the entire region. This represents the attention mechanism.

[0071] Specifically, based on the above attention mechanism, the final feature fusion is performed using the following formula (9): (9) in, Indicates the characteristics after fusion. Indicates the weight of each region. This indicates the characteristics of each region.

[0072] After feature extraction, the display mode is determined based on the extracted features, that is, the display mode is determined based on the recognition result. For this process, the second model is set to output a softmax value through the Classify module after multiple convolutions and attention enhancement. The softmax function is defined by the following formula (10): (10) in, This represents the probability that input x belongs to category y. This represents the score for category y. Indicates the number of categories, such as in binary classification. Take 2. Correspondingly, in When taking 2, .

[0073] Finally, the recognition result of the second model is determined according to the following formula (11): (11) in, This indicates the recognition result of the second model. When the maximum value is obtained When the value is 1, it indicates that the first electronic device is in the first display mode. When the maximum value is obtained When the value of is 0, it indicates that the first electronic device is in the second display mode.

[0074] Therefore, the second model in this application embodiment can support the broad recognition and judgment of different PPT templates and brand projection software (such as Office, WPS, PDF Viewer, etc.), without relying on specific formats or external tags, and has strong generalization ability and adaptability. Furthermore, the model parameters of this second model are controlled within a small range, thus supporting edge computing deployment and can be directly integrated into terminal display devices such as the first electronic device, without relying on cloud resources, thereby achieving real-time, low-latency judgment.

[0075] It is worth noting that the second electronic device can also determine its desired display mode based on the operation performed by the operator and generate a corresponding mode signal to transmit to the first electronic device. The first electronic device receives the mode signal sent by the second electronic device to determine the display mode.

[0076] The second aspect of this application also provides a first electronic device. Since the principle of solving the problem by the electronic device in this application is similar to the information processing method described above, the implementation of the electronic device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0077] Figure 3 A schematic diagram of the structure of the first electronic device provided in an embodiment of this application is shown, with reference to... Figure 3 It can be seen that the first electronic device includes: A communication interface is used to establish a projection channel with a second electronic device; A display screen is used to display image data transmitted through a projection channel; Memory, used to store computer instructions; Processor, for loading the computer instructions and executing at least the following: During the output process based on image data, an input signal is obtained; in response to being in a first display mode, the input signal is responded to based on a first processing mechanism; in response to being in a second display mode, the input signal is responded to based on a second processing mechanism; the electronic devices that generate the output signal corresponding to the input signal under the first processing mechanism and the second processing mechanism are different.

[0078] As another example, the processor is also used to run a second model that extracts multidimensional features from the image data to determine a corresponding display mode.

[0079] In another example, when the processor runs the second model, it specifically performs the following: inputting image data into the second model to obtain the display mode output by the second model; the second model is able to extract multi-dimensional features of the image data to determine the corresponding display mode; wherein, the second model adopts multi-channel input, and the multi-dimensional features include features of different regions in the image.

[0080] In another example, when the processor extracts multidimensional features from the image data to determine the corresponding display mode, it specifically performs the following: extracting features from the first region of the image to identify the title bar; extracting features from the second region of the image to identify the task bar; extracting features from the third region of the image to identify the navigation menu; extracting features from the fourth region of the image to identify the notes bar; and determining the display mode based on the identification results.

[0081] In another example, when the processor responds to the input signal based on the first processing mechanism, it specifically performs the following: generating an output signal based on the input signal; and displaying the output signal.

[0082] In another example, when the processor performs the action of generating an output signal based on an input signal and displaying the output signal, it specifically performs at least one of the following: determining that the input signal represents a finger movement of an operator, generating a first output signal, displaying additional content based on the first output signal, wherein the layer of the additional content is above the layer of the image data; determining that the input signal represents a finger scaling of an operator, generating a second output signal, and adjusting the display size of the image data based on the second output signal.

[0083] In another example, the processor is further configured to perform at least one of the following: in response to being in a first display mode, not responding to an operation type corresponding to the input signal; the operation type including one or more of clicking, touching the entire screen area, and touching a portion of the screen area; in response to being in the first display mode, enabling a third electronic device, establishing a connection with the third electronic device, and displaying prompt content indicating that the operator in the current scene can use the third electronic device to control the display of the first electronic device; in response to being in the first display mode, blocking pop-up messages in the image data; in response to being in the first display mode, invoking a first model to perform a generation task; In another example, when the processor responds to the input signal based on the second processing mechanism, it specifically performs the following: determining the type of the input signal; in response to the input signal being of a first type, transmitting the first input signal to the second electronic device to cause the second electronic device to generate an output signal; in response to the input signal being of a second type, determining a second input signal based on the first input signal, transmitting the second input signal to the second electronic device to cause the second electronic device to generate an output signal; wherein the data formats of the first input signal and the second input signal are different.

[0084] In another example, when the processor executes the operation of determining a second input signal based on the first input signal in response to the input signal being of the second type, it specifically performs the following: parsing the input signal to obtain key value data, converting the key value data into continuous coordinate data; and determining the second input signal based on the continuous coordinate data; the operation corresponding to the second input signal is the same as that corresponding to the first input signal.

[0085] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the storage medium. Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, which will not be repeated here. Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed on a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be executed in a different order than those described here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any specific hardware and software combination.

[0086] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this application that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, which will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0087] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. Rather, the subject matter of this application may be less than all the features of a particular disclosed embodiment. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated as being possible in various combinations or arrangements. The scope of this application should be determined by reference to the appended claims and the full scope of their equivalents.

[0088] The foregoing has described in detail several embodiments of this application, but this application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications based on the concept of this application, and all such variations and modifications should fall within the scope of protection claimed in this application.

Claims

1. An information processing method, applied to a first electronic device, comprising: The input signal is obtained during the output process based on image data; The image data is obtained based on a projection channel between the device and the second electronic device. In response to being in a first display mode, the input signal is responded to based on a first processing mechanism; In response to being in a second display mode, the input signal is responded to based on a second processing mechanism; The electronic devices that generate the output signal corresponding to the input signal under the first processing mechanism and the second processing mechanism are different.

2. The information processing method according to claim 1, wherein responding to the input signal based on a first processing mechanism, includes: Generate an output signal based on the input signal; The output signal is displayed accordingly.

3. The information processing method according to claim 2, wherein an output signal is generated based on an input signal, and the output signal is displayed, includes at least one of the following: The input signal is determined to represent the movement of the operator's finger, a first output signal is generated, and additional content is displayed based on the first output signal, wherein the layer of the additional content is above the layer of the image data; The input signal is determined to represent the scaling of the operator's finger, a second output signal is generated, and the display size of the image data is adjusted based on the second output signal.

4. The information processing method according to claim 1 further includes at least one of the following: In response to being in a first display mode, the operation type corresponding to the input signal is not responded to; the operation type includes one or more of click, touch of the entire screen area, and touch of a portion of the screen area. In response to being in the first display mode, the third electronic device is enabled, a connection is established with the third electronic device, and a prompt is displayed indicating that the operator in the current scene can use the third electronic device to control the display of the first electronic device; In response to being in the first display mode, pop-up messages in the image data are blocked; In response to being in the first display mode, the first model is invoked to perform the generation task.

5. The information processing method according to claim 1, wherein responding to the input signal based on a second processing mechanism includes: Determine the type of the input signal; In response to the input signal being of a first type, the first input signal is transmitted to the second electronic device so that the second electronic device generates an output signal; In response to the input signal being of the second type, a second input signal is determined based on the first input signal, and the second input signal is transmitted to the second electronic device so that the second electronic device generates an output signal; the data formats of the first input signal and the second input signal are different.

6. The information processing method according to claim 5, in response to the input signal being of the second type, determining a second input signal based on the first input signal, includes: The input signal is analyzed to obtain key value data, and the key value data is converted into continuous coordinate data; The second input signal is determined based on the continuous coordinate data; The operation corresponding to the second input signal is the same as that corresponding to the first input signal.

7. The information processing method according to any one of claims 1-6, further comprising: Image data is input into a second model to obtain the display mode output by the second model; the second model can extract multi-dimensional features of the image data to determine the corresponding display mode; wherein, the second model adopts multi-channel input, and the multi-dimensional features include features of different regions in the image.

8. The information processing method according to claim 7, extracting multidimensional features of the image data to determine the corresponding display mode, comprising: Extract features from the first region of the image to identify the title bar; Extract features from the second region of the image and recognize the taskbar; Extract features from the third region of the image to identify the navigation directory; Extract features from the fourth region of the image to identify the remarks column; The display mode is determined based on the recognition results.

9. A first electronic device, comprising: A communication interface is used to establish a projection channel with a second electronic device; A display screen is used to display image data transmitted through a projection channel; Memory, used to store computer instructions; Processor, for loading the computer instructions and executing at least the following: The input signal is obtained during the output process based on image data; In response to being in a first display mode, the input signal is responded to based on a first processing mechanism; In response to being in a second display mode, the input signal is responded to based on a second processing mechanism; The electronic devices that generate the output signal corresponding to the input signal under the first processing mechanism and the second processing mechanism are different.

10. The first electronic device according to claim 9, The processor is also used to run a second model, which extracts multidimensional features from the image data to determine a corresponding display mode.

Images