Intent recognition methods, devices and electronic equipment

Abstract

This application discloses an intent recognition method, apparatus, and electronic device, belonging to the field of artificial intelligence technology. The intent recognition method includes: responding to a user's first input to a first screen area, performing image processing on a first image to obtain a second image; wherein the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image excluding the first area; and recognizing the user's target intent based on the first image feature information of the first image and the second image feature information of the second image.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an intent recognition method, device, and electronic device. Background Technology

[0002] Currently, mainstream screen recognition interaction solutions typically allow users to select areas of interest on the screen by drawing circles or smearing after triggering the screen recognition function. The system responds to the user's selection input by cropping that area from the current screen image and then identifies the user's intent based on the cropped image, providing a basis for subsequent interactive operations. However, the user intent obtained in this way often differs significantly from the user's true intent, resulting in low accuracy. Summary of the Invention

[0003] The purpose of this application is to provide an intent recognition method, apparatus, and electronic device that can accurately recognize a user's intent.

[0004] In a first aspect, embodiments of this application provide an intent recognition method, including: In response to a user's first input to a first screen area, image processing is performed on the first image to obtain a second image; wherein, the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image excluding the first area; Based on the first image feature information of the first image and the second image feature information of the second image, the user's target intent is identified.

[0005] In this embodiment, after receiving the user's first input on the first screen area, the method no longer simply crops the area selected by the user. Instead, it first processes the first image to obtain a second image, making the image feature intensity of the first area selected by the first input in the second image higher than the image feature intensity of the second area not selected by the first input. This achieves the goal of highlighting the area of ​​interest to the user while fully preserving the visual information of the surrounding background. Based on the image before and after processing, the user's intention to select the area can be identified, which can effectively avoid feature loss caused by local cropping, thereby accurately identifying the user's intention.

[0006] In some embodiments of this application, in response to a user's first input to a first screen area, image processing is performed on a first image to obtain a second image, including: In response to the first input, the overlap between the first region and each third region is determined; wherein the third region is the region where each entity object is located in the second image; If the overlap between the first region and all third regions is less than a first threshold and the color histogram entropy of the first region is greater than a second threshold, then image processing is performed on the first image to obtain the second image.

[0007] In this embodiment, after receiving the user's first input, the overlap between the first region and the third regions corresponding to each entity object in the cached first image is first determined. Only when the overlap is less than a first threshold and the first region contains certain visual content will image processing be performed on the first image to highlight the first region and suppress the second region without discarding the second region. In this way, redundant calculations and invalid calculations caused by accidental touches can be avoided, saving resources and improving the accuracy of intent recognition. In some embodiments of this application, the intent recognition method further includes: If the overlap between the first region and at least one third region is greater than or equal to a first threshold, the first intent classification information corresponding to at least one third region is determined as the user's intent. The first intent classification information is obtained by intent recognition based on the first image feature information of the first image.

[0008] In this embodiment, when the first region and the cached third region highly overlap, the user intent corresponding to the cached third region can be directly invoked as the user's intent to select the first region. There is no need to perform repeated calculations, which achieves a "zero-latency" interactive response visually. At the same time, it can avoid the problem of inconsistent output results for the same target caused by the deviation caused by secondary recognition.

[0009] In some embodiments of this application, the intent recognition method further includes: If the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is less than or equal to the second threshold, output the recommendation intent information. The recommendation intent information includes at least one of the following: copying the first image, storing the first image, or sharing the first image.

[0010] In this embodiment, when the overlap between the first region and the third region corresponding to each entity object is low and the first region does not contain visual content, there is no need to perform subsequent reasoning operations. The default recommendation intent information is directly output, which avoids invalid calculations and saves system resources.

[0011] In some embodiments of this application, image processing is performed on the first image to obtain the second image, including: Keep the pixel parameters in the first region unchanged. The pixel parameters include at least one of the following: brightness, contrast, and saturation. According to the first processing mode, the image of the second region is processed to obtain the second image. The first processing mode includes at least one of the following: reducing the brightness of the second region, reducing the transparency of the second region, and displaying a mask with a first transparency in the second region.

[0012] The above processing mode only reduces the visual transparency of the background area, without completely obscuring it. This reduces the interference of the background area on the selected area while preserving the visual information of the background area, providing contextual information for the selected area and helping to improve the accuracy of intent recognition.

[0013] In some embodiments of this application, identifying the user's intent based on first image feature information of a first image and second image feature information of a second image includes: Based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image, the user's intent is identified; The first intent classification information is obtained by performing intent recognition based on the first image feature information of the first image.

[0014] In this embodiment, when recognizing the user's intent on the selected area, in addition to considering the second image feature information and the first image feature information, the first intent classification information is also considered, and it is no longer limited to the single factor of the first image. In this way, the accuracy of intent recognition can be improved.

[0015] In some embodiments of this application, user intent is identified based on first intent classification information, first image feature information of a first image, and second image feature information of a second image, including: The first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information; the user intent information includes the user's intent. In this spatial attention mask image, the pixel value corresponding to the first region is a first gray value; the pixel value corresponding to the second region is a second gray value; and the first gray value is greater than the second gray value.

[0016] This embodiment uses a spatial attention mask image to provide display guidance for the model, forcing the model to focus on the selected area. At the same time, it supplements some lost details by referring to the feature information of the first image, and narrows the scope of intent by using the first intent classification information as prior knowledge. By fusing multiple features, it ensures that the model will not lose context due to cropping, nor will it be distracted by cluttered backgrounds, thus significantly improving the accuracy of intent recognition.

[0017] In some embodiments of this application, the intent recognition model includes: a sampling module, a weighting module, a feature fusion module, and an intent recognition module; The first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information, including: The spatial attention mask image is downsampled using the sampling module to obtain an attention weight map. The spatial size of the attention weight map is the same as the spatial size of the feature information of the second image. The second image feature information and the attention weight map are weighted and fused through the weighting module to obtain the third image feature information; The feature fusion module fuses the third image feature information, the first image feature information, and the first intent classification information to obtain fused feature information. The intent recognition module infers from the fused feature information and outputs the user intent information.

[0018] This embodiment leverages a large model to fuse multiple features, providing both hard spatial guidance from the mask and soft semantic guidance from the intent. At the same time, it utilizes lossless original features to compensate for information that may be lost during image processing, thereby improving the accuracy of intent recognition.

[0019] In some embodiments of this application, the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information, including: The first image feature information, the second image feature information, the spatial attention mask image, and the first intent classification information are input into the intent recognition model to obtain multiple candidate intents and the first confidence level of each candidate intent; Based on global prior information, update the confidence value of the first confidence level for each candidate intent to obtain the second confidence level for each candidate intent; Candidate intents with a second confidence level greater than or equal to the confidence threshold are identified as user intent information. Among them, the global prior information is information associated with the user; the global prior information includes at least one of the following: the user's historical behavior profile information and the user's geographical location information; the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior, and the historical behavior profile information is used to represent the user's historical preference information; the historical preference information is determined based on historical behavior; the user behavior includes the interface operation behavior corresponding to the first input.

[0020] In this embodiment, when recognizing a user's intent, global prior information such as the user's historical behavior profile and geographical location information is introduced to achieve personalized intent recognition. This can provide different intent recognition results for different users and improve the accuracy of intent recognition.

[0021] In some embodiments of this application, the global prior information includes the user's geographic location information; the user's geographic location information includes a first location; the multiple candidate intents include a first candidate intent and a second candidate intent; the first candidate intent is a search object intent; the second candidate intent is an address navigation intent; When the distance between the first position and the second position is less than or equal to the third threshold, the second confidence level corresponding to the second candidate intention is higher than the second confidence level corresponding to the first candidate intention. When the distance between the first position and the second position is greater than the third threshold, the second confidence level corresponding to the first candidate intent is higher than the second confidence level corresponding to the second candidate intent. The second location is the geographical location indicated by the first region.

[0022] This embodiment can dynamically adjust the confidence level of each candidate intent based on the relationship between the user's geographical location and the second location indicated by the first area, so that the final output user intent information matches the user's current scene, thereby improving the user's satisfaction with the intent recognition results.

[0023] In some embodiments of this application, before processing the first image to obtain the second image in response to a user's first input to the first screen area, the intent recognition method further includes: Acquire a first image and contextual information; the contextual information includes at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users; wherein, the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior; the historical behavior profile information is used to characterize the user's historical preference information; the historical preference information is determined based on historical behavior; the user behavior includes the interface operation behavior corresponding to the first input; similar users are other users whose similarity to the user in the first dimension is greater than or equal to the similarity threshold, and the first dimension includes at least one of the following: attribute features, behavior patterns, and contextual situations; Encode the first image to obtain the first image feature information of the first image; The first image feature information, context information, and first prompt word are input into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

[0024] This embodiment can perform intent recognition in advance based on the first image feature information of the first image, the context information associated with the current user, and the first prompt word to obtain the first intent classification information, which is no longer limited to the first image. This improves the recognition accuracy of the first intent classification information and also provides prior information for subsequent deep recognition, ensuring the recognition accuracy of the user's intent.

[0025] In some embodiments of this application, after inputting the first image feature information, context information, and the first prompt word into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information, the intent recognition method further includes: Obtain the parameter extraction model corresponding to the first intent classification information; Input the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information; The extracted result information is the specific content information extracted from the first image based on the first image feature information and the second prompt word; the second prompt word is the prompt word corresponding to the first intent classification information.

[0026] After obtaining the first intent classification information, this embodiment can further combine it with a parameter extraction model to mine the data required to realize the intent from the first image, eliminating the need for users to manually input information and greatly improving interaction efficiency.

[0027] In some embodiments of this application, the intent recognition method further includes: The extracted results information is displayed, including schedule text information. In response to the user's second input, display schedule creation options; In response to the user's selection of schedule creation options, a schedule creation interface is displayed, which includes schedule information extracted from the schedule text information.

[0028] In this embodiment, when the extracted result information includes schedule text information, in response to the user's first input, the schedule creation option can be directly displayed, and based on the user's selection of the schedule creation option, the schedule creation interface including schedule information can be automatically displayed, which simplifies user operation and improves operation efficiency.

[0029] In some embodiments of this application, the number of first intent classification information is at least two; the intent recognition method further includes: Obtain the user's historical preference information; the historical preference information includes at least one of the following: intent classification information, application name; The priority of each first intent classification information is determined based on historical preference information and the first confidence level of each first intent classification information. Output the extraction results for each first intent classification information in descending order of priority.

[0030] After obtaining the extraction results information corresponding to each first intent classification information, this embodiment can further adaptively adjust the confidence level of each first intent classification information based on the user's historical preference information, determine the priority of each first intent classification information based on the adjusted confidence level, and display the extraction results information corresponding to each first intent classification information in descending order of priority, which helps users quickly find the corresponding first intent classification information.

[0031] In some embodiments of this application, after outputting the extraction result information corresponding to each first intent classification information in descending order of priority, the intent recognition method further includes: Receives user input selecting target extraction result information from multiple extraction result information; In response to input selection, the application's interface corresponding to the target extraction result information is displayed.

[0032] This embodiment provides users with an interactive entry point, allowing them to quickly access the next interface without manual input, thus improving user efficiency.

[0033] In some embodiments of this application, the intent recognition method further includes: Display the first intent classification information; If no confirmation input from the user regarding the first intent classification information is received within the first time period, the user's first input to the first screen area is received.

[0034] By using a wait-listen mechanism, users can be given sufficient time to make decisions, and on-demand computation can be achieved. That is, the subsequent intent recognition process will only be started if no user confirmation input is received within the first time period, thus saving computing resources.

[0035] Secondly, embodiments of this application provide an intent recognition device, including: The processing module is used to process the first image in response to the user's first input to the first screen area to obtain the second image; wherein the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area; The recognition module is used to identify the user's target intent based on the first image feature information of the first image and the second image feature information of the second image.

[0036] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.

[0037] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0038] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run programs or instructions to implement the steps of the method as described in the first aspect.

[0039] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which, when executed by at least one processor, implements the steps of the method described in the first aspect.

[0040] Understandably, the electronic device provided in the third aspect, the readable storage medium provided in the fourth aspect, the chip provided in the fifth aspect, and the computer program product provided in the sixth aspect are all used to execute the intent recognition method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0041] Figure 1 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 2 A schematic diagram of the interface of social application A provided for some embodiments of this application; Figure 3 Schematic diagrams of user selection of outer garment entities provided for some embodiments of this application; Figure 4 A schematic diagram of the entity marker box corresponding to the outer shell entity provided in some embodiments of this application; Figure 5 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 6 This application provides schematic diagrams illustrating how a user can trigger the screen recognition function by long-pressing the screen with two fingers, as part of some embodiments of the present application. Figure 7 A schematic diagram corresponding to the trajectory of the first input provided for some embodiments of this application; Figure 8 A schematic diagram of the entity marker box corresponding to the trajectory of the first input provided for some embodiments of this application; Figure 9A schematic diagram of a list of recommended intent options provided for some embodiments of this application; Figure 10 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 11 A schematic diagram of a second image provided for some embodiments of this application; Figure 12 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 13 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 14 Flowcharts of intent recognition methods provided for some embodiments of this application; Figure 15 A schematic diagram illustrating the intended output information for Zhang San provided for some embodiments of this application; Figure 16 A schematic diagram illustrating the intent output information corresponding to Li Si, provided for some embodiments of this application; Figure 17 Illustrations showing user preference for searching for similar applications, provided for some embodiments of this application; Figure 18 This is a schematic diagram illustrating the interface of application D after a user clicks on application D to search for the same product, as provided in some embodiments of this application. Figure 19 A schematic diagram illustrating a user's two-finger long-press on a chat interface, provided for some embodiments of this application; Figure 20 A schematic diagram showing schedule text information in underlined form is provided for some embodiments of this application; Figure 21 A schematic diagram illustrating a list of operating options provided for some embodiments of this application; Figure 22 A schematic diagram of a schedule creation interface provided for some embodiments of this application; Figure 23 A schematic diagram showing the first intent classification information provided for some embodiments of this application; Figure 24 A schematic diagram of the same interface provided for some embodiments of this application; Figure 25 A schematic diagram of a navigation interface provided for some embodiments of this application; Figure 26 Schematic diagrams of the structure of an intent recognition device provided for some embodiments of this application; Figure 27 Schematic diagrams of the structure of electronic devices provided for some embodiments of this application; Figure 28A schematic diagram of the hardware structure of an electronic device provided for some embodiments of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0044] The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. The terminology involved in the embodiments of this application is explained below.

[0045] Entity object: An independent unit that electronic devices identify from an image and that has certain business meaning or operability.

[0046] Entity tag box: A tag box used to mark entity objects. It can be a rectangle, a circle, or other shapes.

[0047] User intent: The purpose or need that a user currently wants to achieve, inferred by analyzing images and user context. For example, in this embodiment of the application, user intent may include intents such as "search for similar products in images", "address navigation", "calendar creation", and "copy text".

[0048] Spatial attention mask image: A single-channel grayscale image of the same size as the original image, used to explicitly guide the model to focus on specific regions of the image at the feature level.

[0049] Intent recognition model: A deep learning model that infers user intent from multi-dimensional information in the input.

[0050] Historical behavior profile information: Through the analysis and refinement of a large number of users' historical interaction behaviors, feature tags or data structures that can represent user preferences are formed.

[0051] Cue words: Instructions written in natural language to guide a model to complete a specific task; usually, they are a piece of text.

[0052] The intent recognition method, apparatus, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0053] The intent recognition method provided in this application can be applied to e-commerce shopping scenarios, life service scenarios, office scenarios, learning assistance scenarios, etc.

[0054] Taking an e-commerce shopping scenario as an example, when a user is browsing application A, they see an outfit picture and think the coat in the picture looks good, but there is no product link in the picture. At this time, the user can trigger the screen recognition function. After the screen recognition function is triggered, the solution of this application embodiment can identify the user's intention to "search for the same style".

[0055] Taking a lifestyle service scenario as an example, a user receives a message in social application B that says "Come to store D in area C for a meal this weekend". The user can long-press the message to trigger the screen recognition function. After the screen recognition function is triggered, the solution in this application embodiment can identify the user's intention to "search for stores" or "navigate".

[0056] The intent recognition method, apparatus, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0057] It should be noted that the intent recognition method provided in this application can be executed by an electronic device with data processing capabilities, such as a mobile phone, tablet computer, laptop computer, or PDA. In some embodiments of this application, the electronic device also has touch screen and screen recognition functions; for example, a user can long-press the screen to trigger the screen recognition function of the electronic device. Some embodiments of this application use an electronic device as the executing entity to illustrate the intent recognition method provided in this application.

[0058] Figure 1 The flowchart illustrates an intent recognition method provided in this application embodiment, which may include the following steps 110-120.

[0059] Step 110: In response to the user's first input to the first screen area, perform image processing on the first image to obtain a second image; wherein, the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area.

[0060] The first input here is used to select an area of ​​interest for the user. For example, this first input may include input from the user selecting the area of ​​interest using a touch device such as a finger or stylus. In some embodiments of this application, the first input may be input from the user circling the area of ​​interest from the click location along a reference direction using their fingertip or knuckle. The reference direction may include a clockwise direction or a counterclockwise direction.

[0061] The first screen area is the area selected by the first input. For example, the first screen area can be any area of ​​the screen, such as a partial area of ​​the entire screen or the entire screen area. That is, the user can select a partial area of ​​the screen as the region of interest, or the entire screen as the region of interest.

[0062] Figure 2 An exemplary schematic diagram of the interface of a social application S is provided. This interface 201 includes a selection of products from the Zhenshang store in area A, such as coats 2011, tables 2012, sofas 2013, trousers 2014, and watches 2015. Users can select areas of interest as needed. Figure 3 Take the example of a user selecting a coat from 2011. Figure 3 The dashed line in the image represents the trajectory corresponding to the first input. In practical applications, this trajectory may not be displayed.

[0063] For example, after receiving the user's first input, the electronic device can display... Figure 4 The solid box 401 shown indicates that the user has selected the outer cover 2011. In practical applications, users can select one or more areas.

[0064] The first image can be a cropped image of the entire screen, preserving complete image information.

[0065] To effectively avoid the loss of some features due to direct cropping of the first screen area, which could affect the accuracy of user intent recognition, image processing can be performed on the first image, for example. For instance, image enhancement processing can be applied to the first region of the first image, while image suppression processing can be applied to the second region; alternatively, the first region can be kept unchanged, while image suppression processing can be applied to the second region, thereby highlighting the first region while fully preserving the second region. The first region is the area selected by the first input, and the second region is the image area excluding the first region, also known as the background region.

[0066] In this embodiment, the image feature intensity of the first region is higher than that of the second region. The image feature intensity may include, but is not limited to, image brightness, image contrast, and image saturation. In this way, the first region can be highlighted, the interference of the second region can be reduced, and the second region can be completely preserved, providing a basis for accurately recognizing the user's intent.

[0067] In some embodiments of this application, the pixel parameters of the first region can remain unchanged, while the pixel parameters of the second region can be reduced, thereby highlighting the first region. In other embodiments of this application, the pixel parameters of the first region can also be increased, while the pixel parameters of the second region can remain unchanged or be reduced. By performing image processing on the first image, the region of interest to the user can be effectively highlighted, while the background region can be completely preserved, providing a reliable foundation for subsequent intent recognition.

[0068] Step 120: Identify the user's intent based on the first image feature information of the first image and the second image feature information of the second image.

[0069] In some embodiments of this application, the first image feature information may be image features obtained by encoding a first image. Similarly, the second image feature information may be image features obtained by encoding a second image.

[0070] In some other embodiments of this application, in order to reduce the transmission time of the first image, the first image can be compressed first to obtain a compressed image, and then the compressed image can be encoded to obtain image features, which are used as the first image feature information of the first image.

[0071] In some other embodiments of this application, after obtaining the compressed image, the compressed image can also be divided into blocks, and then the blocks are encoded to obtain the image features corresponding to each block. The image features corresponding to each block are used as the first image feature information of the first image.

[0072] In some embodiments of this application, the first image feature information obtained above can be stored, and the stored first image feature information can be directly used later without repeating the processing of the first image. This can save computing resources and the processing time of the first image, and improve the efficiency of intent recognition.

[0073] For example, user intent can be identified based on a large model. For instance, first image information and second image feature information can be input into the large model, which then performs inference to identify the user's intent. Because the second image highlights the first region while weakening the second region, the large model can focus on the specific area of ​​interest to the user, avoiding interference from background noise. Simultaneously, the first image feature information contains the original image information, enabling the large model to accurately understand the environment of the first region, thereby outputting an intent judgment result that matches the scenario and improving the accuracy of intent recognition.

[0074] In some embodiments of this application, there may be one or more user intents. When there are multiple user intents, each user intent can be output according to its confidence level. For example, the user intent with higher confidence level will be displayed earlier.

[0075] In this embodiment, after receiving the user's first input on the first screen area, the method no longer simply crops the area selected by the user. Instead, it first processes the first image to obtain a second image, making the image feature intensity of the first area selected by the first input in the second image higher than the image feature intensity of the second area not selected by the first input. This achieves the goal of highlighting the area of ​​interest to the user while fully preserving the visual information of the surrounding background. Based on the image before and after processing, the user's intention to select the area can be identified, which can effectively avoid feature loss caused by local cropping, thereby accurately identifying the user's intention.

[0076] Figure 5 Flowcharts of intent recognition methods provided for some embodiments of this application, Figure 5 and Figure 1 The difference is that, Figure 1 Step 110 in the middle can be refined as follows: Figure 5 Steps 510-520 in the text.

[0077] Step 510: In response to the first input, determine the overlap between the first region and each third region.

[0078] The third region here refers to the area where each entity object in the second image is located. Here, an entity object is an independent unit identified by the electronic device from the first image, possessing a certain business meaning or operability. For example, this entity object may include, but is not limited to, text block entities, product entities, key information entities, schedule entities, avatar entities, etc.

[0079] Text block entities can be text paragraphs with certain semantics, such as "The weather is so nice today, let's go out and enjoy the sun" and "Zhenshang Store in Area A".

[0080] The product entity can be a separately identifiable product item in the first image, such as, but not limited to, "sneakers," "dress," and "beef burger meal." For example, Figure 2 The coat 2011, table 2012, sofa 2013, pants 2014, and watch 2015 are all physical products.

[0081] Key information entities can be structured information extracted from text, such as, but not limited to, time "tomorrow afternoon 3 o'clock", phone number "100-123-45676", address "Unit 101, G Community, N District, M City", etc.

[0082] Schedule entities can be time-related content, such as "10:00-11:30 Project review meeting" or "Buy milk after get off work today".

[0083] A profile picture can be a personal image that represents a user's identity; for example, it can include a round profile picture, a square profile picture, etc.

[0084] For example, after receiving screen-triggered input, the electronic device can capture the entire screen to obtain a first image, and identify all entity objects in the first image and their locations. For example, each identified entity object can be identified using an entity marker box. This entity marker box can be displayed in the background and not directly shown to the user.

[0085] Screen-triggered input can be input executed when the user wakes up the screen recognition function. For example, the screen-triggered input can include a two-finger long press trigger input on the screen, or a single-finger long press trigger input on the screen.

[0086] In some embodiments of this application, such as Figure 6 As shown, after the electronic device receives the user's two-finger long-press trigger input 601 on the screen, it can activate the screen recognition function. In some embodiments of this application, the intent recognition method may further include the following steps: In response to screen-triggered input, a screen animation is displayed to notify the user that the screen-triggered input has been received.

[0087] For example, after receiving a user's screen trigger input, the electronic device can display a preset screen animation to indicate that the screen trigger input has been received. This avoids the user performing repeated operations due to uncertainty about whether the trigger was successful, thereby reducing the ineffective concurrent load on the electronic device. This application does not limit the specific form of the screen animation. For example, the screen animation can be concentric circular ripples spreading outward from the touch point, similar to ripples on water; it can also be a halo effect or a flashing effect; or it can be a bright border fading in and out around the identified entity, flashing in a breathing manner.

[0088] In this embodiment, after the user triggers the screen, the user is informed through visual animation that the touch operation has been received and is being processed. This not only avoids the user from triggering the screen repeatedly due to lack of feedback, reducing invalid requests, but also smooths out the time difference between the technical processing and the user's perception.

[0089] For example, after receiving the user's first input, the electronic device can determine the coordinates of the first region selected by the first input. Based on the coordinates of the first region and the coordinates of the third regions where each entity object is located, the overlap between the first region and each third region can be determined. Taking the third region W as an example, the electronic device can determine the overlapping area of ​​the first region A1 and the third region W, as well as the union area of ​​the first region A1 and the third region W. According to the ratio of the overlapping area to the union area, the overlap between the first region A1 and the third region W can be determined. The union area of ​​the first region A1 and the third region W = the area of ​​the first region A1 + the area of ​​the third region W - the overlapping area. The calculation process for the overlap between the first region A1 and other third regions is similar. Thus, the overlap between the first region and the third region corresponding to each entity object can be obtained. Based on the overlap, it can be determined whether the first region coincides with a previously cached third region. If the currently selected region coincides with a previously cached third region, the intent recognition result corresponding to the cached third region can be directly obtained without performing repeated intent recognition operations, thereby saving computing resources and improving intent recognition efficiency.

[0090] Step 520: When the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is greater than the second threshold, perform image processing on the first image to obtain the second image.

[0091] The values ​​of the first threshold and the second threshold can be set according to actual needs. For example, in some embodiments of this application, the first threshold can be set to 80% and the second threshold can be set to 0.01.

[0092] The fact that the overlap between the first region and all third regions is less than the first threshold indicates that the overlap between the currently selected region and the previously cached third regions is low. In other words, none of the previously cached third regions highly overlap with the currently selected region.

[0093] Color histogram entropy is a metric that measures the richness of colors in an image. The lower the color histogram entropy, the more monotonous and flat the colors in the image. For example, if a user selects a pure white wall or a pure black desktop, this area usually contains no content and is considered an "invalid area." If the color histogram entropy is greater than a second threshold, it means that the area selected by the user contains some visual content.

[0094] When the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is greater than the second threshold, it indicates that the region selected by the user is not in the cache, but there is some visual content in the first region. At this time, image processing and depth recognition need to be performed on the first image.

[0095] In this embodiment, after receiving the user's first input, the overlap between the first region and the third region corresponding to each entity object in the cached first image is first determined. Only when the overlap is less than the first threshold and the first region contains certain visual content will the first image be processed to highlight the first region and suppress the second region, but without discarding the second region. In this way, redundant calculations and invalid calculations caused by accidental touches can be avoided, saving computing resources and improving the recognition accuracy of user intent.

[0096] In some embodiments of this application, the intent recognition method may further include the following steps: If the overlap between the first region and at least one third region is greater than or equal to a first threshold, the first intent classification information corresponding to at least one third region is determined as the user's user intent.

[0097] When the overlap between the first region and the third region of a cached entity object is greater than or equal to a first threshold, it indicates that the first region and the third region of the cached entity object highly overlap. In this case, the first intent classification information corresponding to the third region can be directly determined as the user's intent to select the first region. The first intent classification information is the user's intent for each entity object obtained by analyzing the first image feature information of the first image after the electronic device receives the user's screen trigger input.

[0098] For example, if the overlap between the first region and the third region M is greater than the first threshold, after the electronic device receives the user's screen trigger input, it can determine the intent corresponding to the third region M as "search for similar products in images" by analyzing the first image feature information of the first image. Therefore, it can directly determine that the user's intent in selecting the first region is "search for similar products in images".

[0099] In this embodiment, when the first region and the cached third region highly overlap, the user intent corresponding to the cached third region can be directly invoked as the user's intent to select the first region. There is no need to perform repeated calculations, which achieves a "zero-latency" interactive response visually. At the same time, it can avoid the problem of inconsistent output results for the same target caused by the deviation caused by secondary recognition.

[0100] In some embodiments of this application, the intent recognition method may further include the following steps: If the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is less than or equal to the second threshold, output the recommendation intent information. The recommendation intent information includes at least one of the following: copying the first image, storing the first image, or sharing the first image.

[0101] For example, when the first area does not highly overlap with all the third areas and the first area does not contain visual content, it can be determined that the user is currently making a mistake. At this time, the electronic device can output recommended intent information, which is also the default intent information. For example, it may include, but is not limited to, copying the first image, storing the first image, or sharing the first image.

[0102] For example, such as Figure 7 As shown, the dashed line 701 represents the trajectory corresponding to the user's first input, which is not actually displayed to the user. The electronic device responds to the user's first input and can display... Figure 8 As shown in the figure, the entity marker box 801 is used. The electronic device can then determine the overlap between the area containing entity marker box 801 and the areas containing coat 2011, table 2012, sofa 2013, pants 2014, and watch 2015, as well as the color histogram entropy of the area containing entity marker box 801. After comparison, it is found that the overlap between the area containing entity marker box 801 and the areas containing coat 2011, table 2012, sofa 2013, pants 2014, and watch 2015 is less than the first threshold, and the color histogram entropy of the area containing entity marker box 801 is also less than the second threshold. At this point, it can be determined that the user has accidentally touched the device, and the following output is shown: Figure 9 The recommended intent option list 901 shown does not require further inference operations. This recommended intent option list 901 is exemplified by containing three types of recommended intent information: copy, share, and save screenshot. The screenshot can be the first image. In practical applications, other recommended intent information can also be included.

[0103] Users can then select appropriate recommended intent information as needed. For example, if a user selects the copy intent, the electronic device can directly copy the first image.

[0104] In this embodiment, when the overlap between the first region and the third region corresponding to each entity object is low and the first region does not contain visual content, there is no need to perform subsequent reasoning operations. The default recommendation intent information is directly output, which avoids invalid calculations and saves system resources.

[0105] Figure 10 Flowcharts of intent recognition methods provided for some embodiments of this application, Figure 10 and Figure 1 The difference is that, Figure 1 Step 110 in the middle can be refined as follows: Figure 10 Steps 1000-1010 in the process.

[0106] Step 1000: Keep the pixel parameters in the first region unchanged.

[0107] The pixel parameters here can include at least one of the following: brightness, contrast, and saturation.

[0108] In some embodiments of this application, pixel parameters such as brightness, contrast, and saturation in the first region can be kept unchanged. In this way, the information in the first region can be completely preserved, which helps to improve the accuracy of intent recognition.

[0109] Step 1010: Process the image of the second region according to the first processing mode to obtain the second image.

[0110] The first processing mode can be a processing method that reduces the intensity of image features in the second region, such as including at least one of the following: reducing the brightness of the second region, reducing the transparency of the second region, or displaying a mask with a first transparency in the second region.

[0111] By suppressing the background area, the selected area can be highlighted, but the outline and semantics of the background area are still preserved. This provides a global context for the selected area, ensuring that the intent recognition does not deviate from the real scene, thereby improving the accuracy of intent recognition.

[0112] The specific reduction in brightness and transparency can be flexibly set according to actual needs. The first transparency can be, for example, a value between greater than 0% and less than 100%. For instance, in some embodiments of this application, the first transparency can be 50% transparency, that is, a mask with 50% transparency is displayed in the second area.

[0113] By reducing the image feature intensity of the second region, the selected area can be highlighted, but the outline and semantics of the second region are still preserved. This can provide a global context for the first region, ensuring that the intent recognition does not deviate from the real scene, thereby improving the accuracy of user intent recognition.

[0114] For example, such as Figure 11 As shown, taking the user's selection of four physical objects—jacket 2011, table 2012, sofa 2013, and pants 2014—as an example, after the electronic device receives the user's first input, it can keep the pixel parameters of the areas where jacket 2011, table 2012, sofa 2013, and pants 2014 are located unchanged, while reducing the brightness of the background area.

[0115] The above processing method only reduces the visual transparency of the background area, and does not completely obscure the background area. In this way, it can reduce the interference of the background area on the selected area, while fully preserving the visual information of the background area, providing contextual information for the selected area, and helping to improve the accuracy of user intent recognition.

[0116] Figure 12 Flowcharts of intent recognition methods provided for some embodiments of this application, Figure 12 and Figure 1 The difference is that, Figure 1 Step 120 in the text can be further refined as follows: Figure 12 Step 1200 in the process.

[0117] Step 1200: Identify the user's intent based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image.

[0118] The first intent classification information is obtained by performing intent recognition based on the first image feature information of the first image. The specific process for determining the first intent classification information can be found in the following embodiment.

[0119] The second image feature information provides pixel-level visual input, intuitively telling the model "which area the user has selected," while background suppression preserves the global context "what the surrounding environment is like." The first image feature information provides lossless, original information. While the second image feature information highlights the first region, background suppression may lose some subtle edges or details. Introducing the first image feature information can compensate for information that may be lost during image processing, ensuring the model sees the most original image. The first intent classification information provides prior knowledge, telling the model "what the user probably intended to do initially," which helps narrow down the candidate range when receiving the first input. For example, if the first intent classification information is "address," the model will tend to extract text information from the selected area rather than identifying a product.

[0120] For example, the first intent classification information, the first image feature information, and the second image feature information can be input into a large model, and the large model can perform inference to output the user's intent for the selected area.

[0121] In this embodiment, when recognizing the user's intent on the selected area, in addition to considering the second image feature information and the first image feature information, the first intent classification information is also considered, and it is no longer limited to the single factor of the first image. In this way, the accuracy of recognizing the user's intent can be improved.

[0122] Figure 13 Flowcharts of intent recognition methods provided for some embodiments of this application, Figure 13 and Figure 12 The difference is that, Figure 12 Step 1200 in the middle can be further refined as follows: Figure 13 Step 1300 in the process.

[0123] Step 1300: Input the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information into the intent recognition model to obtain user intent information.

[0124] The spatial attention mask image here is used to visually guide the intent recognition model at the feature extraction level, forcing the model to focus on a first region. The spatial attention mask image is a single-channel grayscale image of the same size as the first image, containing only grayscale shades and no color. For example, all pixels in the first region of the first image can be assigned a first grayscale value, and all pixels in the second region can be assigned a second grayscale value, thus obtaining the spatial attention mask image. The first grayscale value is greater than the second grayscale value. For example, in some embodiments of this application, the first grayscale value can be 255, and the second grayscale value can be 50. A larger grayscale value indicates a higher attention weight.

[0125] The intent recognition model can be a deep learning model that can deeply recognize the user's intent. This embodiment does not limit the specific structure of the intent recognition model.

[0126] For example, the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the second image, and the first intent classification information can be input into the intent recognition model. The intent recognition model performs inference analysis and outputs user intent information for selecting the first region. This user intent information may include the user intent for selecting the first region. In some embodiments, the user intent information may also include the confidence level of the user intent. That is, the intent recognition model can not only output the user intent but also the confidence level of the user intent.

[0127] This embodiment uses a spatial attention mask image to provide display guidance for the model, forcing the model to focus on the selected area. At the same time, it supplements some lost details by referring to the feature information of the first image, and narrows the scope of intent by using the first intent classification information as prior knowledge. By fusing multiple features, it ensures that the model will not lose context due to cropping, nor will it be distracted by cluttered backgrounds, thus significantly improving the accuracy of intent recognition.

[0128] Taking an intent recognition model that includes a sampling module, a weighting module, a feature fusion module, and an intent recognition module as an example, in some embodiments of this application, step 1300 above may include the following steps: The spatial attention mask image is downsampled using a sampling module to obtain an attention weight map; the spatial dimensions of the attention weight map are the same as those of the second image. The second image feature information and the attention weight map are weighted and fused through the weighting module to obtain the third image feature information; The feature fusion module fuses the third image feature information, the first image feature information, and the first intent classification information to obtain fused feature information. The intent recognition module infers from the fused feature information and outputs the user intent information.

[0129] The sampling module is used to reduce the spatial size of the spatial attention mask image. This sampling module can be a downsampling module. By downsampling the spatial attention mask image through the downsampling module, an attention weight map can be obtained. The spatial size of the attention weight map is the same as the spatial size corresponding to the feature information of the second image, which provides a basis for subsequent information fusion.

[0130] The weighting module applies spatial attention weights to the second image feature information to achieve hard guidance. For example, the second image feature information can be multiplied element-wise with the attention weight map to achieve fusion of the two. Since the grayscale value of the selected region is greater than that of the background region, meaning the weight of the selected region is greater than that of the background region, fusing the second image feature information with the attention weight map can further enhance the features of the selected region and further weaken the features of the background region, helping the intent recognition model focus on the region selected by the user.

[0131] The feature fusion module is used to fuse third image feature information, first image feature information, and first intent classification information. To better fuse these three elements, for example, the first intent classification information can first be embedded through an embedding layer to obtain the intent vector corresponding to the first intent classification information. Then, the third image feature information, the first image feature information, and the intent vector corresponding to the first intent classification information are fused.

[0132] For example, the intent vectors corresponding to the third image feature information, the first image feature information, and the first intent classification information can be concatenated into a long vector as fused feature information.

[0133] For example, feature fusion can also be performed using a cross-attention mechanism. Specifically, the feature information of the third image can be treated as a sequence. The first image feature information is used as another set of sequences. The intent vector is projected through a linear layer to align its dimensions with the dimensions of the first image feature information, resulting in an intent query vector. .

[0134] For example, the intent query vector can be As a query, and The concatenated features are used as key and value, and cross-attention is performed to obtain an intent-aware visual feature. This feature emphasizes visual content that is relevant to the intent.

[0135] Similarly, it can be As a query, Using these as key and value, cross-attention is performed to obtain a focused feature with enhanced detail. Similarly, it can be... , and By concatenating or summing element by element, the fused feature information F can be obtained.

[0136] By reasoning through the fused feature information F using the intent recognition module, the user's intent information can be obtained. For example, this intent recognition module may include a fully connected layer and a normalization layer. The fused feature information F is passed sequentially through the fully connected layer and the normalization layer to obtain the user's intent and its corresponding confidence level. For example, the output might be: {"Address Navigation": 0.92, "Search Shops": 0.05, "Other": 0.03}.

[0137] This embodiment leverages a large model to fuse multiple features, providing both hard spatial guidance from the mask and soft semantic guidance from the intent. At the same time, it utilizes lossless original features to compensate for information that may be lost during image processing, thereby improving the accuracy of intent recognition.

[0138] Figure 14 Flowcharts of intent recognition methods provided for some embodiments of this application, Figure 15 and Figure 13 The difference is that, Figure 13 Step 1300 in the middle can be further refined as follows: Figure 14 Steps 1410-1430 in the text.

[0139] Step 1410: Input the first image feature information, the second image feature information, the spatial attention mask image, and the first intent classification information into the intent recognition model to obtain multiple candidate intents and the first confidence level of each candidate intent.

[0140] The specific process of this step can be found in the above embodiments, and will not be repeated here.

[0141] Step 1420: Based on the global prior information, update the confidence value of the first confidence of each candidate intent to obtain the second confidence of each candidate intent.

[0142] The global prior information here refers to information associated with the user. Global prior information may include at least one of the following: the user's historical behavioral profile information, and the user's geographic location information.

[0143] Historical behavior profile information comprises behavioral information related to a user's past actions. User behavior can include interface actions corresponding to the first input. Interface actions are the behaviors that occur after the user performs the first input, such as where they selected an area, what they clicked after selecting, or which app they navigated to. Analyzing current user behavior helps to obtain behavioral information about related behaviors, providing a data foundation for refining user preferences.

[0144] Historical behavior profiles are used to characterize a user's historical preferences, which are determined based on historical behavior. Historical behavior refers to actions that are associated with the current user behavior. This association can be context-dependent; for example, the application scenario in which the historical behavior occurred may be similar to the application scenario corresponding to the current user behavior. For instance, if the current user behavior is: the user long-presses on an outfit image in the QQ app, the electronic device will focus on retrieving the user's past long-press behaviors on outfit-related images in the QQ app or other image-text applications.

[0145] For example, the association here can also be an operation association. For instance, historical behavior can be triggered by the same user operation, such as a two-finger long press or selection. For example, if the current user behavior is: the user triggers screen recognition by a two-finger long press, the electronic device will focus on retrieving a series of operations triggered by the user in the past by the two-finger long press, such as what was clicked and where it was navigated to.

[0146] Analyzing only historical behavior information that is associated with current user behavior can avoid introducing a lot of noise and ensure the quality of global prior information.

[0147] By analyzing users' historical behavioral profiles, we can obtain their historical preferences. These preferences may include, but are not limited to, frequently used app types, operational habits in specific scenarios such as long-pressing to save images, historical search keywords, and preferences for specific content, such as liking to view beauty-related handwriting or frequently browsing technology news. This historical behavioral profile information can be abstracted into vectors or labels using statistical methods and machine learning to describe users' long-term interests and behavioral patterns. Using this historical behavioral profile information as contextual information allows for a more accurate prediction of the user's current true intentions.

[0148] For example, if a user is currently using the A2 app to long-press on a lipstick image with two fingers, their historical behavior profile tags include: preference for makeup, frequent saving of outfit photos, active at night, and recent search for "lipstick swatches". Based on these historical behavior profile tags, we can initially infer that the user's current intention is either "to save this lipstick swatch image" or "to search for the same lipstick".

[0149] The user's geographical location information can be determined based on the device's Global Positioning System (GPS) or Wi-Fi location. It is understandable that, for the same selected area on the same map, different users may have different intentions based on their historical behavior. This embodiment can adjust the first confidence level of each candidate intention based on global prior information, so that the final determined intention conforms to the user's habits.

[0150] For example, an adjustment coefficient or weight for each candidate intent can be determined based on global prior information, and then the coefficient or weight can be multiplied by the first confidence level to obtain the second confidence level for each candidate intent.

[0151] Taking Zhang San and Li Si as examples, suppose both Zhang San and Li Si selected... Figure 2 In the "Zhenshang Store in Area A" section, the selected area corresponds to three candidate intents: {"Address Navigation": 0.7, "Search Store": 0.6, "Copy Address": 0.4}.

[0152] Zhang San's historical behavioral profile includes: frequently using app C to search for shops, never using map navigation, and currently being located in City D, which is very far from District A. Li Si's historical behavioral profile includes: frequently using app M for map navigation, occasionally using app C to search for shops, and currently being located in District A.

[0153] Since Zhang San is in City D, which is far from Area A, his navigation needs should be significantly reduced; and since Zhang San habitually uses app C to search for shops, his shop search needs should be significantly increased. Based on this, we can determine that the adjustment coefficient for the address navigation intent is 0.1, the adjustment coefficient for the shop search intent is 1.5, and the adjustment coefficient for the address copy intent is 1. Therefore, for Zhang San, the adjusted confidence levels for each candidate intent are: {"Search for shops": 0.90, "Copy address": 0.40, "Address navigation": 0.07}.

[0154] Since Li Si is located in Area A, his navigation needs are strong, and he frequently uses navigation services. Therefore, his navigation needs should be significantly increased, while his store search needs can be considered secondary and retained appropriately. Based on this, we can determine that the adjustment coefficient for the address navigation intent is 1.8, the adjustment coefficient for the store search intent is 0.8, and the adjustment coefficient for the address copy intent is 1. Thus, for Li Si, the adjusted confidence levels for each candidate intent are: {"Address navigation": 1.26, "Search store": 0.48, "Copy address": 0.40}.

[0155] Step 1430: Determine the candidate intents with a second confidence level greater than or equal to the confidence level threshold as the user's user intent information.

[0156] For example, user intent information can be determined from each candidate intent based on the adjusted confidence level. For instance, candidate intents with a second confidence level greater than or equal to the confidence level threshold can be determined as user intent information, or all candidate intents can be used as user intent information and arranged in descending order of confidence level.

[0157] For example, such as Figure 15 and Figure 16 As shown, Figure 15 Output information 1501 for Zhang San's intent. That is, for the same selected area 1500 of the same image, the output user intent information is: search for store, copy address, address navigation. Figure 16 The corresponding intent output information for Li Si is 1601, which means that for the same selected area 1500 of the same image, the output user intent information is: address navigation, search for stores, copy address.

[0158] In this embodiment, when recognizing a user's intent, global prior information such as the user's historical behavior profile and geographical location information is introduced to achieve personalized intent recognition. This can provide different intent recognition results for different users and improve the accuracy of intent recognition.

[0159] Taking the global prior information including the user's geographic location information, the user's geographic location information including the first location, and multiple candidate intents including the first candidate intent and the second candidate intent, the first candidate intent being the intent to search for an object and the second candidate intent being the intent to navigate to an address as an example, for instance, when the distance between the first location and the second location is less than or equal to the third threshold, the second confidence level corresponding to the second candidate intent is higher than the second confidence level corresponding to the first candidate intent; When the distance between the first position and the second position is greater than the third threshold, the second confidence level corresponding to the first candidate intent is higher than the second confidence level corresponding to the second candidate intent. The second location is the geographical location indicated by the first region.

[0160] For example, if the distance between the first location and the second location is less than or equal to the third threshold, it means that the user is closer to the second location. In this case, the second confidence level corresponding to the second candidate intention is higher than the second confidence level corresponding to the first candidate intention. That is, the user is more likely to navigate to the second location.

[0161] For example, if the distance between the first position and the second position is greater than the third threshold, it means that the user is far away from the second position. In this case, the second confidence level corresponding to the second candidate intent is lower than the second confidence level corresponding to the first candidate intent, that is, the user is more likely to search.

[0162] For example, when both intents are displayed, the intent with higher confidence is placed first.

[0163] This embodiment can dynamically adjust the confidence level of each candidate intent based on the relationship between the user's geographical location and the second location indicated by the first area, so that the final output user intent information matches the user's current scene, thereby improving the user's satisfaction with the intent recognition results.

[0164] The process of determining the first intent classification information is explained in detail below.

[0165] In some embodiments of this application, prior to step 110, the intent recognition method may further include the following steps: Obtain a first image and contextual information; the contextual information includes at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users; wherein, similar users are other users whose similarity to the user in the first dimension is greater than or equal to the similarity threshold, and the first dimension includes at least one of the following: attribute features, behavior patterns, and contextual situations; Encode the first image to obtain the first image feature information of the first image; The first image feature information, context information, and first prompt word are input into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

[0166] For example, after receiving screen-triggered input, the electronic device can acquire a first image. The screen-triggered input is used to activate the screen recognition function of the electronic device, and may include, for example, two-finger long-press input, single-finger long-press input, etc.

[0167] In some embodiments of this application, to reduce transmission time, the electronic device, upon receiving screen-triggered input from a user, can acquire a first image and compress it. For example, the first image can be compressed proportionally to a certain range. For instance, if the resolution of the first image is 1080*2400, assuming the longer side is compressed from 2400 to 1600, the scaling ratio is 1600 / 2400 = 0.667. Based on this scaling ratio, the shorter side can be compressed to 1080*0.667 = 720.

[0168] For example, after receiving screen-triggered input from a user, the electronic device can also obtain contextual information associated with the user, which may include at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users.

[0169] Similar users here can be other users whose similarity to the current user in the first dimension is greater than or equal to the similarity threshold. The size of the similarity threshold can be set according to actual needs, for example, it can be set to a value between 80% and 100%. For example, in some embodiments of this application, the similarity threshold can be set to 90%.

[0170] The first dimension may include at least one of the following: attribute characteristics, behavioral patterns, and contextual situations. Attribute characteristics can be relatively static, inherent attributes representing a user, such as, but not limited to, demographic attributes, device attributes, and geographic attributes. Users with similar attributes often have similar basic needs. Demographic attributes may include, for example, age, gender, and occupation. Device attributes may include, for example, device model, operating system model and version. Geographic attributes may include, for example, the city where the user resides.

[0171] Behavioral patterns are used to characterize the stable habits and patterns that users exhibit during their interactions with the system. These patterns may include, but are not limited to, commonly used app types, frequently used functions, click preferences, active time periods, and consumption habits.

[0172] Contextual information is used to characterize the environmental state in which a user performs an action. This may include, but is not limited to, the specific application currently being used, the type of content currently displayed on the screen, the time of day, whether the user is on the move, and the network environment.

[0173] The interaction behavior information of similar users may include at least one of the following: click behavior, browsing behavior, search behavior, purchase behavior, and dwell behavior generated by similar users within a historical period. In some embodiments, the interaction behavior information of similar users within a historical period may also be analyzed to determine the preference information of similar users within that historical period, providing support for subsequently identifying the intent of the current user.

[0174] For example, when there is limited historical behavior data for the current user, interaction behavior information from similar users can be considered. By analyzing the interaction behavior of similar users, the problem of sparse historical data for the current user can be compensated for, and potential intent trends can also be discovered.

[0175] For example, if a user long-presses an image in app P, the electronic device finds several similar users. After long-pressing an image in a similar scenario, 80% of these similar users clicked "Search for similar items," 15% clicked "Favorite," and 5% clicked "Share." Based on these statistics, the subsequent large model can tilt the current user's intent probability towards "Search for similar items," and then combine this with the current user's own historical preferences to determine the final intent, such as "Search for similar products in shopping app T."

[0176] For example, the first image can be encoded to obtain the first image feature information. Alternatively, the first image can be divided into blocks to obtain multiple block images, and each block image can be encoded separately to obtain the image block features corresponding to each block image. Based on the image block features, the first image feature information of the first image can be obtained. For example, the extraction of features from each image block can be performed in parallel, thus improving the efficiency of image feature extraction. For example, the electronic device can cache the encoded first image feature information, and subsequently, features can be directly retrieved from the cache without re-encoding the first image, significantly saving model inference time.

[0177] The first cue word here guides the intent recognition model. For example, the first cue word may include information such as role setting, task description, candidate intent list, and output format requirements. Role setting tells the intent recognition model its current role, such as being a screen intent recognition expert. Task description tells the intent recognition model what to do. Candidate intent list provides possible options, constraining the output range of the intent recognition model. Output format requirements tell the intent recognition model in what format to return the answer.

[0178] Taking a user long-pressing an image of a model wearing a coat with two fingers on app A as an example, the first prompt word could be as follows: You are a screen intent recognition assistant. The user is using a two-finger long press on the screen in the current application. Based on the screen image content, the user's historical behavior, and the interaction behavior of similar users, determine the user's most likely intent. Please select one of the following options and indicate its confidence level: A. Save the image B. Search for similar products C. Copy image link D. Share with friends Please output in JSON format, for example: {"intent": "B", "confidence": 0.85}.

[0179] For example, the first image feature information, context information, and the first prompt word can be input into the intent recognition model. The intent recognition model then performs inference to obtain the first intent classification information and the first confidence level of the first intent classification information. Referring to the first prompt word given above, the final output of the intent recognition model can be in the following form: {"intent": "B", "confidence": 0.92}.

[0180] The intent recognition model used for the first intent classification information can be the same as the intent recognition model used for the first input.

[0181] This embodiment can perform intent recognition in advance based on the first image feature information of the first image, the context information associated with the current user, and the first prompt word to obtain the first intent classification information, which is no longer limited to the first image. This improves the recognition accuracy of the first intent classification information and also provides prior information for subsequent deep recognition, ensuring the recognition accuracy of the user's intent.

[0182] In some embodiments of this application, after inputting the first image feature information, context information, and the first prompt word into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information, the intent recognition method may further include the following steps: Obtain the parameter extraction model corresponding to the first intent classification information; Input the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information; The extracted result information is the specific content information extracted from the first image based on the first image feature information and the second prompt word; the second prompt word is the prompt word corresponding to the first intent classification information.

[0183] The parameter extraction model is used to extract specific information needed for the first intent classification information from the first image. For example, if the first intent classification information is "address recognition", the corresponding parameter extraction model can extract address information from the first image. For example, if the first image is a street view containing the text "Zhī Cì F Dì Zhāng Dān" and the first intent classification information is "address recognition", the electronic device can retrieve a model specifically for address parameter extraction from the model library to extract the complete address information "Zhī Cì F Dì Zhāng Dān" from the first image.

[0184] In practical applications, the first intent classification information may include, but is not limited to, image search for similar items intent, text search for products intent, schedule intent, address intent, problem-solving intent, etc., specifically related to the image content of the first image. This embodiment only obtains the parameter extraction model corresponding to the first intent classification information. For example, if the first intent classification information corresponding to the first image only includes three intents: image search for similar items, text search for products, and address intent, then only the parameter extraction models corresponding to these three intents are called. This reduces the invalid execution of unnecessary parameter extraction models, saving system resources and memory.

[0185] When there are multiple primary intent classification information, each parameter extraction model can be executed in parallel, thereby reducing online processing time and improving intent recognition efficiency.

[0186] The parameter extraction model takes the first image feature information and the second prompt word obtained in the above embodiments as input, and the extraction result of the first intent classification information as output. Each parameter extraction model has its own unique second prompt word.

[0187] For example, the second prompt for the image search intent is "Please search for similar products based on the image content and return product links" or "Please combine the user's frequency of use of each application and return which application the user prefers to use to search for similar products based on the image." The second prompt for the calendar intent is "Please extract the calendar information from the image, including time, location, and event title." The second prompt for the address intent is "Please identify the address in the image and parse it into structured information."

[0188] The extracted result information can be specific content information extracted from the first image. For example, for a schedule intent, the extracted specific content information may include time, location, event title, etc. For an address intent, the extracted specific content information may include Unit P, Building 2, Apartment 101, S Community, Y District, City X. For a problem-solving intent, the extracted specific content information may include specific problem-solving steps.

[0189] For example, the electronic device can display the extraction result information corresponding to the first intent classification information.

[0190] Taking the example of searching for similar items based on images, where the second suggestion is "Please consider the user's frequency of use of various applications and return which application the user prefers to use for searching for similar items based on images," for instance, as follows: Figure 17 As shown, electronic devices can display which application a user prefers to use to search for similar items for each physical object. For example, for "Sofa 2013," users tend to use app B to search for similar items; for "Coat 2011" and "Table 2012," users tend to use app A; for "Pants 2014," users tend to use app C; and for "Watch 2015," users tend to use app D.

[0191] Figure 17 Taking the display of 5 entity objects as an example, in actual applications, more entity objects can be displayed. The primary intent classification information, confidence level, and extraction result information corresponding to each entity object can be stored together and used directly later.

[0192] If the currently displayed results match the user's needs, the user can click on the corresponding extracted result information to jump to the corresponding interface. For example, if the user clicks "Search for similar products using App D," the electronic device can directly jump to the following page. Figure 18 The interface 1801 of application D shown can display watches found in application D that are the same model as watch 2015: brand F watch 1802 and new men's minimalist niche watch 1803.

[0193] After obtaining the first intent classification information, this embodiment can further combine it with a parameter extraction model to mine the data required to realize the intent from the first image, eliminating the need for users to manually input information and greatly improving interaction efficiency.

[0194] In some embodiments of this application, the intent recognition method may further include the following steps: The extracted results information is displayed, including schedule text information. In response to the user's second input, display schedule creation options; In response to the user's selection of schedule creation options, a schedule creation interface is displayed, which includes schedule information extracted from the schedule text information.

[0195] The second input is used to control or instruct the electronic device to display schedule creation options. Exemplarily, the second input may include touch input, voice input, gesture input, etc. In some embodiments of this application, the second input may be a user's click input of schedule text information.

[0196] The selection input here is used to select the schedule creation option. For example, the selection input can include clicking the schedule creation option, long-pressing the option, etc. Clicking the option can include single-clicking, double-clicking, etc.

[0197] like Figure 19 As shown, taking the user's two-finger long press on the chat interface 1901 of the social application M as an example, after the electronic device receives the user's two-finger long press input on the current chat interface 1901, it can identify the user's intent as "schedule intent" in the manner described in the above embodiment. Then, combined with the parameter extraction model corresponding to the schedule intent, the schedule time can be determined: tomorrow afternoon at 2:30 pm.

[0198] For example, such as Figure 20 As shown, the electronic device can display the specific time of the schedule in the form of an underlined 2001: Tomorrow at 2:30 PM. When the user clicks on the underlined 2001, the following will be displayed: Figure 21 The operation option list 2101 shown may include, for example, a schedule creation option 2102 and a recommended option 2103. The recommended option 2103 may include options such as copy, select all, share, and word segmentation. When a user clicks the schedule creation option 2102, as shown... Figure 22 As shown, a schedule creation interface 2201 can be displayed. Through this schedule creation interface 2201, users can create schedules to avoid forgetting them.

[0199] The schedule creation interface 2201 includes schedule information extracted from the schedule text information. For example, based on the current date and the specific time of the schedule, the start time of the schedule can be determined. For instance, if the current date is February 26, 2026, the start time of the schedule can be determined to be 14:30 on February 27, 2026, without requiring manual input from the user. The end time can be filled in by the user or filled in by the system default. The reminder time can be set to 5 minutes by default, that is, reminding the user 5 minutes in advance, or it can be changed by the user as needed. The reminder method can use the default method or be customized by the user, for example, the default reminder method is an alarm clock reminder. The schedule topic can be obtained from the text information, and... Figure 19 Taking the schedule shown as an example, we can determine that the schedule theme is "Weekly Meeting". Figure 22 This is merely an illustrative diagram of a schedule creation interface; in actual use, the interface may contain other information.

[0200] In this embodiment, when the extracted result information includes schedule text information, in response to the user's first input, the schedule creation option can be directly displayed, and based on the user's selection of the schedule creation option, the schedule creation interface including schedule information can be automatically displayed, which simplifies user operation and improves operation efficiency.

[0201] Taking the example that the number of first intent classification information is at least two, the intent recognition method may further include the following steps: Obtain the user's historical preference information; the historical preference information includes at least one of the following: intent classification information, application name; The priority of each first intent classification information is determined based on historical preference information and the first confidence level of each first intent classification information. Output the extraction results for each first intent classification information in descending order of priority.

[0202] For example, electronic devices can analyze a user's historical behavior and use intent category information to determine the most frequently used intent category information. For instance, if a user is viewing outfit images in app H and 80% of their long-press actions in the past have triggered "search for similar items in images," it can be determined that the user's preferred intent category information is "search for similar items in images."

[0203] For example, electronic devices can analyze the names of applications used by users in their historical behavior to identify the applications that users frequently use. For instance, after triggering the "search for similar items in images" feature, in 90% of cases, the user will choose to navigate to application C rather than application D or application E. Therefore, it can be determined that the user frequently uses application C in their historical behavior.

[0204] For example, an electronic device can determine the priority of each first intent category based on the user's historical preference information and the first confidence level of each first intent category. For example, the weight of each first intent category can be determined based on the user's historical preference information, and the final confidence level of each first intent category can be determined based on this weight and the corresponding first confidence level. The priority of each first intent category is then determined based on the final confidence level. Higher confidence levels result in higher priority.

[0205] Taking a user's experience with the H-view app as an example, analysis of the user's historical behavior reveals that "search for similar images" was triggered most frequently (50%), followed by "address recognition" (30%). Assuming the primary intent category includes "search for similar images (confidence 0.6)" and "address recognition (confidence 0.85), and given the high frequency of "search for similar images" in the user's historical behavior, its weight can be determined to be 0.2 (this is just an example). The adjusted confidence for "search for similar images" is 0.2 + 0.6 = 0.8. Other primary intent categories are processed similarly; based on the final confidence of each category, its priority can be determined.

[0206] For example, the extraction results information corresponding to each first intent classification information can be displayed in descending order of priority.

[0207] Taking "same image search" as an example, it is also possible to analyze the frequency of using different applications to "search for the same image" in the user's historical behavior, and display the extracted results information corresponding to the first intent category information in order of usage frequency from high to low.

[0208] For example, such as Figure 23 As shown, taking "Zhenshang Store in Area A" 1500 as an example, the frequency of users searching for the same product using application C is higher than the frequency of users searching for the same product using application D, and the frequency of users searching for the same product using application D is higher than the frequency of users searching for the same product using application E. Therefore, for "Zhenshang Store in Area A" 1500, the extraction result information 2301 corresponding to each first intent category information can be displayed in the order from top to bottom: searching for the same product using application C, searching for the same product using application D, and searching for the same product using application E.

[0209] After obtaining the extraction results information corresponding to each first intent classification information, this embodiment can further adaptively adjust the confidence level of each first intent classification information based on the user's historical preference information, determine the priority of each first intent classification information based on the adjusted confidence level, and display the extraction results information corresponding to each first intent classification information in descending order of priority, which helps users quickly find the corresponding first intent classification information.

[0210] In some embodiments of this application, after outputting the extraction result information corresponding to each first intent classification information in descending order of priority, the intent recognition method may further include the following steps: Receives user input selecting target extraction result information from multiple extraction result information; In response to the selected input, the interface of the application corresponding to the target extraction result information is displayed.

[0211] The selection input here is used to select target extraction result information. For example, this selection input may include input from a user using a touch device such as a finger or stylus to select the target extraction result information. For instance, the selection input may be a user's click input on the target extraction result information. The target extraction result information can be any one of multiple extraction result information sets.

[0212] For example, after receiving the user's selection input for the target extraction result information, the electronic device can display the program interface of the application corresponding to the target extraction result information to provide the user with the required information.

[0213] For example, such as Figure 23 As shown, assuming a user clicks "Search for similar items in app C", the electronic device, in response to the user's click input, can display something like this. Figure 24 The interface 2401 shown can display items found in application C that are identical to those from "Area A Zhenshang Store". This interface 2401 may include a text search entry 2402, through which users can manually enter the information they wish to search for. The text search entry 2402 displays "Area A Zhenshang Store" by default.

[0214] For example, such as Figure 16 As shown, when a user clicks "Address Navigation," the electronic device responds to the user's input by displaying, as shown below. Figure 25 The navigation interface 2501 shown pre-loads the navigation destination "Zhenshang Store in Area A". In actual application, the user can also manually modify the navigation destination. Based on the navigation destination, the electronic device can display the distance between the user's current location and the navigation destination, as well as the specific location of the navigation destination. For example, after the user clicks the confirmation control 2503, the electronic device can further display the navigation route. After the user clicks the cancel control 2502, they can exit the navigation interface 2501 and return to the previous interface.

[0215] This embodiment provides users with an interactive entry point, allowing them to quickly access the next interface without manual input, thus improving user efficiency.

[0216] In some embodiments of this application, after inputting the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information, the intent recognition method may further include the following steps: Display the first intent classification information; If no confirmation input from the user regarding the first intent classification information is received within the first time period, the user's first input to the first screen area is received.

[0217] For example, when none of the initial intent classification information meets the user's needs, the user can perform a first input on the first screen area. For example, if no confirmation input from the user regarding the initial intent classification information is received within a first time period, it indicates that the user is dissatisfied with the currently displayed initial intent classification information, meaning that the currently displayed initial intent classification information does not meet the user's needs. In this case, the user can perform a first input on the first screen area, and the electronic device will continue to perform deeper intent recognition. This wait-and-listen mechanism provides the user with sufficient decision-making time and enables on-demand computation; that is, the subsequent intent recognition process will only be initiated if no confirmation input from the user is received within the first time period, thus saving computational resources.

[0218] The following example illustrates the complete process of intent recognition.

[0219] Step 2701: Receive screen-triggered input from the user.

[0220] Step 2702: In response to screen-triggered input, display screen animation, and obtain the first image and context information associated with the user.

[0221] Step 2703: Input the first image feature information, context information and the first prompt word into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

[0222] Step 2704: Obtain the parameter extraction model corresponding to the first intent classification information.

[0223] Step 2705: Input the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information.

[0224] Step 2706: Determine the priority of each first intent category based on the user's historical preference information and the first confidence level of each first intent category.

[0225] Step 2708: Output the extraction results for each first intent classification information in order of priority.

[0226] Step 2709: Is the user satisfied? If the user is satisfied, end directly; otherwise, proceed to step 2710.

[0227] Step 2710: Receive the user's first input to the first screen area.

[0228] Step 2711: In response to the first input, determine the overlap between the first region and each third region.

[0229] Step 2712: Check whether the overlap between the first region and all third regions is less than the first threshold and whether the color histogram entropy of the first region is greater than the second threshold. If both are satisfied, proceed to step 2713; otherwise, proceed to step 2714.

[0230] Step 2713: Perform image processing on the first image to obtain the second image.

[0231] Step 2714: Does the overlap between the first region and at least one third region exceed or equal to the first threshold? If yes, proceed to step 2715; otherwise, proceed to step 2716.

[0232] Step 2715: Determine the first intent classification information corresponding to at least one third region as the user's user intent.

[0233] Step 2716: Is the entropy of the color histogram of the first region less than or equal to the second threshold? If yes, proceed to step 2717; otherwise, proceed to step 2713.

[0234] Step 2717: Output recommendation intent information.

[0235] Step 2718: Generate a spatial attention mask image corresponding to the first image.

[0236] Step 2719: Input the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information into the intent recognition model to obtain user intent information.

[0237] In this embodiment, when a user selects a first screen area, a strategy of highlighting the selected area and suppressing the background area is employed to preserve the complete background region. This processing method effectively avoids feature loss caused by local cropping, allowing the model to utilize the background environment as necessary context. Simultaneously, by combining spatial attention mask images and first intent classification information, multiple measures are taken to significantly improve the accuracy of intent recognition in complex scenes. By adjusting only the visual weights of the image, the models used for both intent recognition operations can be unified, avoiding logical conflicts between multiple models and reducing deployment and maintenance costs.

[0238] It should be noted that the intent recognition method provided in this application embodiment can be executed by an intent recognition device or a processing module within that intent recognition device for executing the intent recognition method. This application embodiment uses the execution of the intent recognition method by an intent recognition device as an example to illustrate the intent recognition device provided in this application embodiment.

[0239] Figure 26 This is a schematic diagram of the structure of an intent recognition device provided in an embodiment of this application.

[0240] like Figure 26 As shown, the intent recognition device 2600 may include: Processing module 2601 is used to process the first image in response to the user's first input to the first screen area to obtain a second image; wherein the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area; The recognition module 2602 is used to recognize the user's intent based on the first image feature information of the first image and the second image feature information of the second image.

[0241] In this embodiment, after receiving the user's first input on the first screen area, the method no longer simply crops the area selected by the user. Instead, it first processes the first image to obtain a second image, making the image feature intensity of the first area selected by the first input in the second image higher than the image feature intensity of the second area not selected by the first input. This achieves the goal of highlighting the area of ​​interest to the user while fully preserving the visual information of the surrounding background. Based on the image before and after processing, the user's intention to select the area can be identified, which can effectively avoid feature loss caused by local cropping, thereby accurately identifying the user's intention.

[0242] In some possible implementations of the embodiments of this application, the processing module 2601 is specifically used for: In response to the first input, the overlap between the first region and each third region is determined; wherein the third region is the region where each entity object is located in the second image; If the overlap between the first region and all third regions is less than a first threshold and the color histogram entropy of the first region is greater than a second threshold, then image processing is performed on the first image to obtain the second image.

[0243] In some possible implementations of the embodiments of this application, the processing module 2601 is further configured to determine the first intent classification information corresponding to at least one third region as the user's user intent when the overlap between the first region and at least one third region is greater than or equal to a first threshold. The first intent classification information is obtained by intent recognition based on the first image feature information of the first image.

[0244] In some possible implementations of the embodiments of this application, the processing module 2601 is further configured to output recommendation intent information when the overlap between the first region and all third regions is less than a first threshold and the color histogram entropy of the first region is less than or equal to a second threshold. The recommendation intent information includes at least one of the following: copying the first image, storing the first image, or sharing the first image.

[0245] In some possible implementations of the embodiments of this application, the processing module 2601 is specifically used for: Keep the pixel parameters in the first region unchanged. The pixel parameters include at least one of the following: brightness, contrast, and saturation. According to the first processing mode, the image of the second region is processed to obtain the second image. The first processing mode includes at least one of the following: reducing the brightness of the second region, reducing the transparency of the second region, and displaying a mask with a first transparency in the second region.

[0246] In some possible implementations of the embodiments of this application, the identification module 2602 is specifically used for: Based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image, the user's intent is identified; The first intent classification information is obtained by performing intent recognition based on the first image feature information of the first image.

[0247] In some possible implementations of the embodiments of this application, the identification module 2602 is specifically used for: The first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information; the user intent information includes the user's intent. In this spatial attention mask image, the pixel value corresponding to the first region is a first gray value; the pixel value corresponding to the second region is a second gray value; and the first gray value is greater than the second gray value.

[0248] In some possible implementations of the embodiments of this application, the identification module 2602 is specifically used for: The spatial attention mask image is downsampled to obtain an attention weight map, the spatial size of which is the same as the spatial size of the feature information of the second image; The second image feature information and the attention weight map are weighted and fused to obtain the third image feature information; The third image feature information, the first image feature information, and the first intent classification information are fused to obtain fused feature information; Inference is performed on the fused feature information to output user intent information.

[0249] In some possible implementations of the embodiments of this application, the identification module 2602 is specifically used for: The first image feature information, the second image feature information, the spatial attention mask image, and the first intent classification information are input into the intent recognition model to obtain multiple candidate intents and the first confidence level of each candidate intent; Based on global prior information, update the confidence value of the first confidence level for each candidate intent to obtain the second confidence level for each candidate intent; Candidate intents with a second confidence level greater than or equal to the confidence threshold are identified as user intent information. Among them, the global prior information is information associated with the user; the global prior information includes at least one of the following: the user's historical behavior profile information and the user's geographical location information; the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior, and the historical behavior profile information is used to represent the user's historical preference information; the historical preference information is determined based on historical behavior; the user behavior includes the interface operation behavior corresponding to the first input.

[0250] In some possible implementations of the embodiments of this application, the global prior information includes the user's geographic location information; the user's geographic location information includes a first location; the multiple candidate intents include a first candidate intent and a second candidate intent; the first candidate intent is a search object intent; the second candidate intent is an address navigation intent; When the distance between the first position and the second position is less than or equal to the third threshold, the second confidence level corresponding to the second candidate intention is higher than the second confidence level corresponding to the first candidate intention. When the distance between the first position and the second position is greater than the third threshold, the second confidence level corresponding to the first candidate intent is higher than the second confidence level corresponding to the second candidate intent. The second location is the geographical location indicated by the first region.

[0251] In some possible implementations of this application's embodiments, the processing module 2601 is further configured to acquire the first image and context information before performing image processing on the first image in response to the user's first input to the first screen area to obtain the second image; the context information includes at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users; wherein, the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior; the historical behavior profile information is used to characterize the user's historical preference information; the historical preference information is determined based on historical behavior; the user behavior includes the interface operation behavior corresponding to the first input; similar users are other users whose similarity to the user in the first dimension is greater than or equal to a similarity threshold, and the first dimension includes at least one of the following: attribute features, behavior patterns, and contextual situations; Encode the first image to obtain the first image feature information of the first image; The recognition module 2602 is specifically used for: The first image feature information, context information, and first prompt word are input into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

[0252] In some possible implementations of the embodiments of this application, the processing module 2601 is further configured to, after inputting the first image feature information, context information and the first prompt word into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information, obtain the parameter extraction model corresponding to the first intent classification information; input the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information; The extracted result information is the specific content information extracted from the first image based on the first image feature information and the second prompt word; the second prompt word is the prompt word corresponding to the first intent classification information.

[0253] In some possible implementations of the embodiments of this application, the intent recognition device 2600 further includes: The display module is used to display the extracted result information, including schedule text information; in response to a second user input, it displays schedule creation options; in response to the user's selection of schedule creation options, it displays the schedule creation interface, which includes schedule information extracted from the schedule text information.

[0254] In some possible implementations of the embodiments of this application, the number of first intent classification information is at least two; the processing module 2601 is further configured to: Obtain the user's historical preference information; the historical preference information includes at least one of the following: intent classification information, application name; The priority of each first intent classification information is determined based on historical preference information and the first confidence level of each first intent classification information. Output the extraction results for each first intent classification information in descending order of priority.

[0255] In some possible implementations of the embodiments of this application, the processing module 2601 is further configured to receive the user's selection input of the target extraction result information among the multiple extraction result information after outputting the extraction result information corresponding to each first intent classification information in order of priority from high to low. The display module is also used to display the application's interface corresponding to the target extraction result information in response to the selected input.

[0256] In some possible implementations of the embodiments of this application, the display module is further configured to display first intent classification information; The processing module 2601 is also configured to receive the user's first input to the first screen area if no confirmation input of the user on the first intent classification information is received within the first time period.

[0257] The intent recognition device in this application embodiment can be a device or a component in an electronic device, such as an integrated circuit or a chip. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0258] The electronic device in this application embodiment can be a terminal with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0259] The intent recognition device provided in this application embodiment can achieve... Figures 1 to 25The various processes in the intent recognition method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0260] like Figure 27 As shown, this application embodiment also provides an electronic device 27001, including a processor 27002 and a memory 27003. The memory 27003 stores programs or instructions that can run on the processor 27002. When the program or instructions are executed by the processor 27002, they implement the various steps of the above-described intent recognition method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0261] It should be noted that the electronic devices in the embodiments of this application include the mobile terminals and non-mobile terminals mentioned above.

[0262] Figure 28 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0263] The electronic device 2800 includes, but is not limited to, components such as: radio frequency unit 2801, network module 2802, audio output unit 2803, input unit 2804, sensor 2805, display unit 2806, user input unit 2807, interface unit 2808, memory 2809, and processor 2810.

[0264] Those skilled in the art will understand that the electronic device 2800 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 2810 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 28 The structure of the electronic device 2800 shown does not constitute a limitation on the electronic device 2800. The electronic device 2900 may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be described in detail here.

[0265] The processor 2810 is configured to process the first image in response to a first input from a user to a first screen area to obtain a second image; wherein the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area; and the user's intention is identified based on the first image feature information of the first image and the second image feature information of the second image.

[0266] In this embodiment, after receiving the user's first input on the first screen area, the method no longer simply crops the area selected by the user. Instead, it first processes the first image to obtain a second image, making the image feature intensity of the first area selected by the first input in the second image higher than the image feature intensity of the second area not selected by the first input. This achieves the goal of highlighting the area of ​​interest to the user while fully preserving the visual information of the surrounding background. Based on the image before and after processing, the user's intention to select the area can be identified, which can effectively avoid feature loss caused by local cropping, thereby accurately identifying the user's intention.

[0267] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: In response to the first input, the overlap between the first region and each third region is determined; wherein the third region is the region where each entity object is located in the second image; If the overlap between the first region and all third regions is less than a first threshold and the color histogram entropy of the first region is greater than a second threshold, then image processing is performed on the first image to obtain the second image.

[0268] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: If the overlap between the first region and at least one third region is greater than or equal to a first threshold, the first intent classification information corresponding to at least one third region is determined as the user's intent. The first intent classification information is obtained by intent recognition based on the first image feature information of the first image.

[0269] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: If the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is less than or equal to the second threshold, output the recommendation intent information. The recommendation intent information includes at least one of the following: copying the first image, storing the first image, or sharing the first image.

[0270] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: Keep the pixel parameters in the first region unchanged. The pixel parameters include at least one of the following: brightness, contrast, and saturation. According to the first processing mode, the image of the second region is processed to obtain the second image. The first processing mode includes at least one of the following: reducing the brightness of the second region, reducing the transparency of the second region, and displaying a mask with a first transparency in the second region.

[0271] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: Based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image, the user's intent is identified; The first intent classification information is obtained by performing intent recognition based on the first image feature information of the first image.

[0272] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: The first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information; the user intent information includes the user's intent. In this spatial attention mask image, the pixel value corresponding to the first region is a first gray value; the pixel value corresponding to the second region is a second gray value; and the first gray value is greater than the second gray value.

[0273] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: The spatial attention mask image is downsampled to obtain an attention weight map, the spatial size of which is the same as the spatial size of the feature information of the second image; The second image feature information and the attention weight map are weighted and fused to obtain the third image feature information; The third image feature information, the first image feature information, and the first intent classification information are fused to obtain fused feature information; Inference is performed on the fused feature information to output user intent information.

[0274] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: The first image feature information, the second image feature information, the spatial attention mask image, and the first intent classification information are input into the intent recognition model to obtain multiple candidate intents and the first confidence level of each candidate intent; Based on global prior information, update the confidence value of the first confidence level for each candidate intent to obtain the second confidence level for each candidate intent; Candidate intents with a second confidence level greater than or equal to the confidence threshold are identified as user intent information. Among them, the global prior information is information associated with the user; the global prior information includes at least one of the following: the user's historical behavior profile information and the user's geographical location information; the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior, and the historical behavior profile information is used to represent the user's historical preference information; the historical preference information is determined based on historical behavior; the user behavior includes the interface operation behavior corresponding to the first input.

[0275] In some possible implementations of the embodiments of this application, the global prior information includes the user's geographic location information; the user's geographic location information includes a first location; the multiple candidate intents include a first candidate intent and a second candidate intent; the first candidate intent is a search object intent; the second candidate intent is an address navigation intent; When the distance between the first position and the second position is less than or equal to the third threshold, the second confidence level corresponding to the second candidate intention is higher than the second confidence level corresponding to the first candidate intention. When the distance between the first position and the second position is greater than the third threshold, the second confidence level corresponding to the first candidate intent is higher than the second confidence level corresponding to the second candidate intent. The second location is the geographical location indicated by the first region.

[0276] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: Before processing the first image to obtain the second image in response to the user's first input to the first screen area, the first image and context information are acquired. The context information includes at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users. The historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior. The historical behavior profile information is used to characterize the user's historical preference information. The historical preference information is determined based on historical behavior. The user behavior includes the interface operation behavior corresponding to the first input. Similar users are other users whose similarity to the user in the first dimension is greater than or equal to a similarity threshold. The first dimension includes at least one of the following: attribute features, behavior patterns, and contextual situations. Encode the first image to obtain the first image feature information of the first image; The first image feature information, context information, and first prompt word are input into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

[0277] In some possible implementations of embodiments of this application, the processor 2810 is specifically used for: After inputting the first image feature information, context information, and the first prompt word into the intent recognition model, and obtaining the first intent classification information and the first confidence level of the first intent classification information, Obtain the parameter extraction model corresponding to the first intent classification information; Input the first image feature information and the second prompt word into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information; The extracted result information is the specific content information extracted from the first image based on the first image feature information and the second prompt word; the second prompt word is the prompt word corresponding to the first intent classification information.

[0278] In some possible implementations of the embodiments of this application, the display unit 2806 is specifically used for: The extracted results information is displayed, including schedule text information. In response to the user's second input, display schedule creation options; In response to the user's selection of schedule creation options, a schedule creation interface is displayed, which includes schedule information extracted from the schedule text information.

[0279] In some possible implementations of the embodiments of this application, the number of first intent classification information is at least two; the processor 2810 is specifically used for: Obtain the user's historical preference information; the historical preference information includes at least one of the following: intent classification information, application name; The priority of each first intent classification information is determined based on historical preference information and the first confidence level of each first intent classification information. Output the extraction results for each first intent classification information in descending order of priority.

[0280] In some possible implementations of the embodiments of this application, the user input unit 2807 is used to receive the user's selection input of the target extraction result information among multiple extraction result information after the processor 2810 outputs the extraction result information corresponding to each first intent classification information in order of priority from high to low. The display unit 2806 is specifically used to: display the program interface of the application corresponding to the target extraction result information in response to the selected input.

[0281] In some possible implementations of the embodiments of this application, the display unit 2806 is specifically used for: Display the first intent classification information; User input unit 2807 is specifically used for: If no confirmation input from the user regarding the first intent classification information is received within the first time period, the user's first input to the first screen area is received.

[0282] It should be understood that, in this embodiment, the input unit 2804 may include a graphics processing unit (GPU) 28041 and a microphone 28042. The GPU 28041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 2806 may include a display panel 28061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 2807 includes at least one of a touch panel 28071 and other input devices 28072. The touch panel 28071 is also called a touch screen. The touch panel 28071 may include a touch detection device and a touch controller. Other input devices 28072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0283] The memory 2809 can be used to store software programs and various data. The memory 2809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 2809 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 2809 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0284] Processor 2810 may include one or more processing units; optionally, processor 2810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 2810.

[0285] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described intent recognition method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0286] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0287] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described intent recognition method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0288] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0289] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the intent recognition method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0290] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0291] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0292] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An intent recognition method, characterized in that, include: In response to a user's first input to a first screen area, image processing is performed on a first image to obtain a second image; wherein, the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area; Based on the first image feature information of the first image and the second image feature information of the second image, the user's user intent is identified.

2. The method according to claim 1, characterized in that, The step of responding to a user's first input to a first screen area, performing image processing on a first image to obtain a second image, includes: In response to the first input, the overlap between the first region and each third region is determined; wherein, the third region is the region where each entity object in the second image is located; If the overlap between the first region and all third regions is less than a first threshold and the color histogram entropy of the first region is greater than a second threshold, then image processing is performed on the first image to obtain the second image.

3. The method according to claim 2, characterized in that, The method further includes: If the overlap between the first region and at least one third region is greater than or equal to a first threshold, the first intent classification information corresponding to the at least one third region is determined as the user's intent. The first intent classification information is obtained by intent recognition based on the first image feature information of the first image.

4. The method according to claim 2, characterized in that, The method further includes: If the overlap between the first region and all third regions is less than the first threshold and the color histogram entropy of the first region is less than or equal to the second threshold, output recommendation intent information. The recommendation intent information includes at least one of the following: copying the first image, storing the first image, or sharing the first image.

5. The method according to claim 1, characterized in that, The step of processing the first image to obtain the second image includes: Keep the pixel parameters in the first region unchanged, and the pixel parameters include at least one of the following: brightness, contrast, and saturation; According to the first processing mode, the image of the second region is processed to obtain the second image. The first processing mode includes at least one of the following: reducing the brightness of the second region, reducing the transparency of the second region, and displaying a mask with a first transparency in the second region.

6. The method according to claim 1, characterized in that, The step of identifying the user's intent based on the first image feature information of the first image and the second image feature information of the second image includes: Based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image, the user's intent is identified; The first intent classification information is obtained by performing intent recognition based on the first image feature information of the first image.

7. The method according to claim 6, characterized in that, The step of identifying the user's intent based on the first intent classification information, the first image feature information of the first image, and the second image feature information of the second image includes: The first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information are input into the intent recognition model to obtain user intent information; the user intent information includes the user's user intent. Wherein, the pixel value corresponding to the first region in the spatial attention mask image is a first gray value; the pixel value corresponding to the second region in the spatial attention mask image is a second gray value; and the first gray value is greater than the second gray value.

8. The method according to claim 7, characterized in that, The intent recognition model includes: a sampling module, a weighting module, a feature fusion module, and an intent recognition module; The step of inputting the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information into the intent recognition model to obtain user intent information includes: The spatial attention mask image is downsampled using the sampling module to obtain an attention weight map, the spatial size of which is the same as the spatial size of the second image feature information. The weighting module performs weighted fusion of the second image feature information and the attention weight map to obtain the third image feature information. The feature fusion module performs fusion processing on the third image feature information, the first image feature information, and the first intent classification information to obtain fused feature information. The intent recognition module infers from the fused feature information and outputs user intent information.

9. The method according to claim 7, characterized in that, The step of inputting the first image feature information of the first image, the second image feature information of the second image, the spatial attention mask image corresponding to the first image, and the first intent classification information into the intent recognition model to obtain user intent information includes: The first image feature information, the second image feature information, the spatial attention mask image, and the first intent classification information are input into the intent recognition model to obtain multiple candidate intents and the first confidence level of each candidate intent; Based on global prior information, update the confidence value of the first confidence level for each candidate intent to obtain the second confidence level for each candidate intent; Candidate intents with a second confidence level greater than or equal to the confidence threshold are identified as the user intent information of the user; Wherein, the global prior information is information associated with the user; the global prior information includes at least one of the following: the user's historical behavior profile information, the user's user geographic location information; the historical behavior profile information is behavioral information of historical behaviors associated with the user's user behavior, and the historical behavior profile information is used to characterize the user's historical preference information; the historical preference information is determined based on the historical behavior; the user behavior includes the interface operation behavior corresponding to the first input.

10. The method according to claim 9, characterized in that, The global prior information includes the user's geographic location information; the user's geographic location information includes a first location; the multiple candidate intents include a first candidate intent and a second candidate intent; the first candidate intent is a search object intent; the second candidate intent is an address navigation intent; When the distance between the first position and the second position is less than or equal to the third threshold, the second confidence level corresponding to the second candidate intention is higher than the second confidence level corresponding to the first candidate intention. When the distance between the first position and the second position is greater than the third threshold, the second confidence level corresponding to the first candidate intention is higher than the second confidence level corresponding to the second candidate intention. The second location is the geographical location indicated by the first area.

11. The method according to any one of claims 1-10, characterized in that, Before performing image processing on the first image in response to a user's first input to the first screen area to obtain the second image, the method further includes: The first image and context information are obtained; the context information includes at least one of the following: the application name of the application to which the first image belongs, the user's historical behavior profile information, and the interaction behavior information of similar users; wherein, the historical behavior profile information is the behavior information of historical behaviors associated with the user's user behavior; the historical behavior profile information is used to characterize the user's historical preference information; the historical preference information is determined based on the historical behavior; the user behavior includes the interface operation behavior corresponding to the first input; the similar users are other users whose similarity to the user in a first dimension is greater than or equal to a similarity threshold, and the first dimension includes at least one of the following: attribute features, behavior patterns, and contextual situations; The first image is encoded to obtain the first image feature information of the first image; The first image feature information, the context information, and the first prompt word are input into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information.

12. The method according to claim 11, characterized in that, After inputting the first image feature information, the context information, and the first prompt word into the intent recognition model to obtain the first intent classification information and the first confidence level of the first intent classification information, the method further includes: Obtain the parameter extraction model corresponding to the first intent classification information; The first image feature information and the second prompt word are input into the parameter extraction model to obtain the extraction result information corresponding to the first intent classification information; The extraction result information is the specific content information extracted from the first image based on the first image feature information and the second prompt word; the second prompt word is the prompt word corresponding to the first intent classification information.

13. The method according to claim 12, characterized in that, The method further includes: The extracted result information is displayed, including schedule text information; In response to the user's second input, schedule creation options are displayed; In response to the user's selection of the schedule creation option, a schedule creation interface is displayed, which includes schedule information extracted from the schedule text information.

14. The method according to claim 12, characterized in that, The number of the first intent classification information is at least two; the method further includes: Obtain the user's historical preference information; the historical preference information includes at least one of the following: intent classification information, application name; Based on the historical preference information and the first confidence level of each first intent classification information, the priority of each first intent classification information is determined; Output the extraction results for each first intent classification information in descending order of priority.

15. The method according to claim 14, characterized in that, After outputting the extraction results for each first intent classification information in descending order of priority, the method further includes: Receive the user's selection input of target extraction result information from multiple extraction result information; In response to the selection input, the application interface corresponding to the target extraction result information is displayed.

16. The method according to claim 12, characterized in that, The method further includes: Display the first intent classification information; If no confirmation input from the user regarding the first intent classification information is received within the first time period, the user's first input regarding the first screen area is received.

17. An intent recognition device, characterized in that, include: The processing module is configured to respond to a first input from a user to a first screen area, perform image processing on the first image to obtain a second image; wherein the image feature intensity of the first area in the second image is higher than the image feature intensity of the second area in the second image; the first area is the image area selected by the first input, and the second area is the image area in the second image other than the first area; The recognition module is used to identify the user's target intent based on the first image feature information of the first image and the second image feature information of the second image.

18. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1-16.

Images