Scene recognition method and device for cloud applications
By identifying text from cloud application screen and determining text aggregation area based on their positional relationship and rectangular area height conditions, the problem of low cloud application scenario recognition efficiency in the prior art is solved, and a more efficient recognition process is achieved.
Patent Information
- Application Number
- CN202411123476.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-15
AI Technical Summary
The existing cloud application scenario recognition scheme compares image similarity, resulting in low recognition efficiency and cannot effectively improve the efficiency of cloud application scenario recognition.
By identifying text from cloud application screen, the text aggregation area is determined based on the positional relationship of the text and the height conditions of the rectangular area, thereby performing target scene recognition.
The efficiency of cloud application scenario recognition is improved, and compared with traditional image similarity comparison methods, the recognition speed and accuracy are significantly improved.
Smart Images

Figure CN118629049B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud computing, and in particular to a scene recognition method and device for cloud applications. Background Art
[0002] With the development of the cloud application industry (such as cloud games), more and more applications are running on cloud platforms. Scene recognition during the user's use of cloud applications has become an important part of daily monitoring and analysis, such as identifying the user's login method (free login or authentication login), whether the application is successfully entered, and the pop-up window that appears during the use of cloud applications. Most of the existing cloud application scene recognition solutions determine whether a specific scene appears during the operation of the cloud application by comparing the similarity between the cloud application screen rendered by the cloud application and the cloud application screen in a specific scene. Since this process involves a large number of image similarity comparisons, the recognition efficiency of this solution is low.
[0003] Therefore, how to provide a scene recognition solution for cloud applications to improve the efficiency of cloud application scene recognition has become a technical problem that needs to be solved urgently. Summary of the invention
[0004] In view of the technical problems existing in the prior art, the embodiments of the present application provide a scene recognition method and device for cloud applications.
[0005] In a first aspect, an embodiment of the present application provides a scene recognition method for a cloud application, comprising:
[0006] Acquire a cloud application screen, perform text recognition on a designated area in the cloud application screen, and if text is recognized, perform text area recognition according to the position of the recognized text, wherein at least one text area is recognized when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition;
[0007] If at least one text area is identified, a text aggregation area is determined according to the at least one text area, and a target scene recognition is performed on the cloud application according to the position of the text aggregation area in the cloud application screen.
[0008] In a second aspect, an embodiment of the present application further provides a scene recognition device for a cloud application, including:
[0009] a text recognition unit, used to obtain a cloud application screen, perform text recognition on a designated area in the cloud application screen, and if text is recognized, perform text area recognition according to the position of the recognized text, wherein at least one text area is recognized when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition;
[0010] The scene recognition unit is used to determine a text aggregation area according to the at least one text area if at least one text area is recognized, and perform target scene recognition on the cloud application according to the position of the text aggregation area in the cloud application screen.
[0011] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the scene recognition method for cloud applications as described in the first aspect are executed.
[0012] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the scene recognition method for cloud applications as described in the first aspect.
[0013] In summary, the scene recognition method and device, electronic device and storage medium for cloud applications provided in the embodiments of the present application perform scene recognition based on the layout of text recognized from the cloud application screen. Compared with the existing technology of performing scene recognition by comparing the similarity between images, this solution can improve the efficiency of cloud application scene recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 A flowchart of an embodiment of a scene recognition method for a cloud application provided in an embodiment of the present application;
[0015] Figure 2 A schematic diagram of a cloud application screen of another embodiment of a scene recognition method for a cloud application provided in an embodiment of the present application;
[0016] Figure 3 A schematic diagram of a cloud application screen according to another embodiment of a scene recognition method for a cloud application provided in an embodiment of the present application;
[0017] Figure 4 A schematic diagram of a cloud application screen according to another embodiment of a scene recognition method for a cloud application provided in an embodiment of the present application;
[0018] Figure 5 A schematic diagram of the structure of an embodiment of a scene recognition device for a cloud application provided in an embodiment of the present application;
[0019] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0020] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.
[0021] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0022] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0023] Reference Figure 1 FIG. 1 is a flow chart of a scene recognition method for a cloud application provided in an embodiment of the present application. The scene recognition method for a cloud application includes:
[0024] S10, obtaining a cloud application screen, performing text recognition on a designated area in the cloud application screen, and if text is recognized, performing text area recognition according to the position of the recognized text, wherein when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition, at least one text area is recognized;
[0025] In the present embodiment, it should be noted that the designated area is a partial area or the entire area of the cloud application screen, and the specific demarcation range is determined according to the cloud application scenario to be identified. For example, if the cloud application is XXX and the target scenario to be identified is authentication login, the designated area may be the middle area or the entire area which occupies 3 / 4 of the area of the entire cloud application screen; for example, if the cloud application is YYY and the target scenario to be identified is entering the application, the designated area may be the upper right corner area which occupies 1 / 4 of the area of the entire cloud application screen; for another example, if the cloud application is ZZZ and the target scenario to be identified is a pop-up window, the designated area may be the middle area which occupies 3 / 5 of the area of the entire cloud application screen.
[0026] It should be noted that after recognizing text on the cloud application screen, it is necessary to determine whether the position of the recognized text satisfies specific positional relationship conditions and / or whether the rectangular area to which the recognized text belongs satisfies specific height conditions. Only when the position of the recognized text satisfies specific positional relationship conditions and / or the rectangular area to which the recognized text belongs satisfies specific height conditions, can at least one text area be obtained. Otherwise, it means that the corresponding cloud application screen is not a cloud application screen under a specific scenario, and the subsequent steps will no longer be processed for the corresponding cloud application screen. Instead, the next frame of the cloud application screen will be processed. In other words, the text area of the recognized text is not necessarily the text area to be obtained in the end.
[0027] S11. If at least one text area is identified, a text aggregation area is determined according to the at least one text area, and a target scene is identified for the cloud application according to a position of the text aggregation area in the cloud application screen.
[0028] In this embodiment, it should be noted that the text aggregation area needs to cover all the identified text areas, for example, it can be the smallest area that can cover all the identified text areas. After determining the text aggregation area, it can be determined whether a specific scene appears during the operation of the cloud application based on the distance between the text aggregation area and the left and right boundaries and the upper and lower boundaries of the cloud application screen. For example, if the cloud application is XXX and the target scene to be identified is authentication login, then when the text aggregation area is located in the middle of the cloud application screen, it is determined that the corresponding user has logged in through authentication login; for example, if the cloud application is YYY and the target scene to be identified is entering the application, then when the text aggregation area is located in the upper right corner of the cloud application screen, it is determined that the corresponding user has entered the application; for another example, if the cloud application is ZZZ and the target scene to be identified is a pop-up window, then when the text aggregation area is located in the middle of the cloud application screen, it is determined that a pop-up window appears during the use of the cloud application by the corresponding user.
[0029] The scene recognition method for cloud applications provided in the embodiment of the present application performs scene recognition based on the layout of text recognized from the cloud application screen. Compared with the existing technology of performing scene recognition by comparing the similarity between images, this solution can improve the efficiency of cloud application scene recognition.
[0030] Based on the above method embodiment, if the target scenario is authentication login, the text area recognition according to the position of the recognized text may include:
[0031] If the position of the first type of characters and the position of the second type of characters satisfy a first position relationship, then both the area of the first type of characters and the area of the second type of characters are used as one of the at least one character area; or
[0032] If the position of the first category of characters and the position of the second category of characters satisfy a first position relationship, and the position of the first category of characters and the position of the third category of characters satisfy a second position relationship, then the area of the first category of characters, the area of the second category of characters and the area of the third category of characters are all used as one of the at least one character area, wherein the first position relationship is that the distance between the position of the first category of characters and the position of the second category of characters belongs to a first distance range, and the angle formed by the position of the first category of characters, the position of the second category of characters and the coordinate axis belongs to a first angle range, and the second position relationship is that the distance between the position of the first category of characters and the position of the third category of characters belongs to a second distance range, and the angle formed by the position of the first category of characters, the position of the third category of characters and the coordinate axis belongs to a second angle range.
[0033] In this embodiment, it should be noted that for the authentication login scenario, there are three types of text (recorded as class t1, class t2 and class t3): the keywords of class t1 can include at least one of mobile phone number, user, account, account number and email address; the keywords of class t2 can include at least one of verification code and password; the keywords of class t3 can include at least one of login, registration and next step. The first, second and third categories are different and are all one of class t1, class t2 and class t3. Figure 2 The cloud application screen shown is used as an example to illustrate the identification process of the authentication login scenario. Figure 2 This is a schematic diagram of a frame of a cloud application screen during authentication and login scene recognition. Figure 2In the example, it is assumed that three text regions A, B and C are identified, and the characters in region A, region B and region C are the second type of characters, the first type of characters and the third type of characters respectively. The center point of region A, the center point of region B and the center point of region C are used as the position of the characters in region A, the position of the characters in region B and the position of the characters in region C respectively. Then the length d1 of the line segment formed by the center point of region A and the center point of region B is the distance between the position of the first type of characters and the position of the second type of characters, and the angle 1 is the angle formed by the position of the first type of characters, the position of the second type of characters and the coordinate axis; the length d2 of the line segment formed by the center point of region B and the center point of region C is the distance between the position of the first type of characters and the position of the third type of characters, and the angle 2 is the angle formed by the position of the first type of characters, the position of the third type of characters and the coordinate axis. If d1 belongs to the first distance range, angle 1 belongs to the first angle range, d2 belongs to the second distance range, and angle 2 belongs to the second angle range, it means that the cloud application screen may be a cloud application screen in the authentication login scenario. Afterwards, it is necessary to determine the text aggregation area (i.e., the area framed by the dotted box) based on area A, area B, and area C, and determine whether the text aggregation area is located at a specific position in the cloud application screen. If the text aggregation area is located at a specific position in the cloud application screen, it means that the corresponding user has logged in through authentication login during the use of the cloud application. For example, the absolute value of the difference between the ratio of the distance l1 from the text aggregation area to the left border of the cloud application screen and the width of the cloud application screen, and the ratio of the distance l2 from the text aggregation area to the right border of the cloud application screen and the width of the cloud application screen is less than a preset value, and the absolute value of the difference between the ratio of the distance h1 from the text aggregation area to the upper border of the cloud application screen and the height of the cloud application screen, and the ratio of the distance h2 from the text aggregation area to the lower border of the cloud application screen and the height of the cloud application screen is less than a preset value, then it means that the corresponding user has logged in through authentication login during the use of the cloud application. It should be noted that the first distance range, the first angle range, the second distance range, the second angle range, and the specific position are determined according to the position and layout of the login area in the cloud application screen in the cloud application authentication login scenario. In addition, it should be noted that Figure 2 The authentication login scenario recognition process of the cloud application screen shown identifies three types of text. In actual applications, the cloud application screen in the authentication login scenario of some cloud applications does not necessarily contain three types of text. For example, the cloud application screen in the authentication login scenario of a certain cloud application contains mobile phone number and next step, that is, it only contains two types of text, class t1 and class t3. At this time, when identifying the authentication login scenario, you only need to identify these two types of text and determine whether the position of the text satisfies the corresponding positional relationship.
[0034] Based on the above method embodiment, if the target scene is successfully entering the application, the text area recognition according to the position of the recognized text may include:
[0035] If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, the area of the first group of numbers and the area of the second group of numbers are both used as one of the at least one text area; and / or
[0036] If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, and the position of the first group of numbers and the position of the third group of numbers satisfy a fourth position relationship, the area of the first group of numbers, the area of the second group of numbers and the area of the third group of numbers are all used as one of the at least one text area, wherein the third position relationship is that the distance between the position of the first group of numbers and the position of the second group of numbers belongs to a third distance range, and the angle formed by the position of the first group of numbers, the position of the second group of numbers and the coordinate axis belongs to a third angle range, and the fourth position relationship is that the distance between the position of the first group of numbers and the position of the third group of numbers belongs to a fourth distance range, and the angle formed by the position of the first group of numbers, the position of the third group of numbers and the coordinate axis belongs to a fourth angle range.
[0037] In this embodiment, it should be noted that for a successful entry into an application scenario, a certain frame of a cloud application screen is usually rendered in this scenario, and a specific area of the cloud application screen (such as the upper right corner area) usually displays at least one set of numbers, which are used to represent the various application tokens, experience points, etc. owned by the user. Therefore, if the target scenario is to successfully enter the application, the text recognition process for the specified area in the cloud application screen recognizes numbers.
[0038] Specifically, if the specific area displays only one group of numbers, and the recognition result is one group of numbers, the area of the group of numbers is used as the at least one text area; if the specific area displays two groups of numbers, and the recognition result is two groups of numbers (assuming the first group of numbers and the second group of numbers), it is necessary to determine whether the position of the first group of numbers and the position of the second group of numbers satisfy the third position relationship. If so, the area of the first group of numbers and the area of the second group of numbers are both used as one text area in the at least one text area; if the specific area displays three groups of numbers, and the recognition result is three groups of numbers (assuming the first group of numbers, the second group of numbers and the third group of numbers), it is necessary to determine whether the position of the first group of numbers and the position of the second group of numbers satisfy the third position relationship, and whether the position of the first group of numbers and the position of the third group of numbers satisfy the fourth position relationship. If the position of the first group of numbers and the position of the second group of numbers satisfy the third position relationship, and the position of the first group of numbers and the position of the third group of numbers satisfy the fourth position relationship, the area of the first group of numbers, the area of the second group of numbers and the area of the third group of numbers are all used as one text area in the at least one text area.
[0039] Assume that a specific area displays three sets of numbers. Figure 3 The cloud application screen shown is used as an example to illustrate the identification process of successfully entering the application scenario. Figure 3 This is a schematic diagram of a frame of a cloud application screen when successfully entering the application scene recognition. Figure 3 In , it is assumed that three text regions D, E and F are identified, and the characters in region D, region E and region F are the second group of numbers, the first group of numbers and the third group of numbers respectively. The center point of region D, the center point of region E and the center point of region F are respectively used as the position of the second group of numbers, the position of the first group of numbers and the position of the third group of numbers. Then the length d3 of the line segment formed by the center point of region D and the center point of region E is the distance between the position of the first group of numbers and the position of the second group of numbers, and the angle 3 is the angle formed by the position of the first group of numbers, the position of the second group of numbers and the coordinate axis; the length d4 of the line segment formed by the center point of region E and the center point of region F is the distance between the position of the first group of numbers and the position of the third group of numbers, and the angle 4 is the angle formed by the position of the first group of numbers, the position of the third group of numbers and the coordinate axis. If the distance d3 belongs to the third distance range, the angle 3 belongs to the third angle range, and the distance d4 belongs to the fourth distance range, and the angle 4 belongs to the fourth angle range, then region D, region E and region F are the three identified text regions, and the area covering region D, region E and region F (i.e. Figure 3The area enclosed by the dashed box in the middle is the text aggregation area. Then, it is determined whether the text aggregation area is located at a specific position in the cloud application screen. If the text aggregation area is located at a specific position in the cloud application screen, it means that the corresponding user has successfully entered the application. For example, the absolute value of the difference between the ratio of the distance l3 from the text aggregation area to the left border of the cloud application screen and the width of the cloud application screen, and the ratio of the distance l4 from the text aggregation area to the right border of the cloud application screen and the width of the cloud application screen is greater than a preset value, and the absolute value of the difference between the ratio of the distance h4 from the text aggregation area to the lower border of the cloud application screen and the height of the cloud application screen, and the ratio of the distance h3 from the text aggregation area to the upper border of the cloud application screen and the height of the cloud application screen is greater than a preset value, it means that the corresponding user has entered the application. The third distance range, the third angle range, the fourth distance range, the fourth angle range and the specific position are determined according to the position and layout of the application tokens / experience points in the cloud application screen that is usually rendered when the user successfully enters the application scenario.
[0040] It should be noted that, in this embodiment, only the text area recognition logic for the situations where one, two, and three groups of application tokens / experience are displayed in specific positions on the cloud application screen when the application scenario is successfully entered is listed, which basically covers the cloud applications available on the market. The core logic is to count the position characteristics and layout characteristics of the numbers on the cloud application screen that are usually rendered when the application scenario is successfully entered based on the position and layout of the numbers on the cloud application screen that are usually rendered when the application scenario is successfully entered, and then identify whether the corresponding user has successfully entered the cloud application by judging whether the cloud application screen has this feature when identifying the successful entry into the application scenario.
[0041] On the basis of the foregoing method embodiment, if the target scene is a pop-up window, the text area recognition according to the position of the recognized text may include:
[0042] Determine the rectangular area to which the recognized text belongs according to the position of the recognized text, and obtain at least one rectangular area Q;
[0043] If it is determined that the number of the at least one rectangular area Q is 1 and the height of the at least one rectangular area Q belongs to the first height range, or the number of the at least one rectangular area Q is not 1 and the height of the at least one rectangular area Q belongs to the first height range and any two adjacent rectangular areas in the at least one rectangular area Q satisfy the fifth positional relationship, at least one rectangular area in the at least one rectangular area Q is used as a reference, and a target rectangular area is searched upward and downward on the cloud application screen respectively, wherein the fifth positional relationship is that the longitudinal distance between the two rectangular areas belongs to the first longitudinal distance range, the target rectangular area is at least one rectangular area in the rectangular areas to which other texts except the recognized text belong in the cloud application screen, the height of the target rectangular area belongs to the first height range or the second height range or the third height range, the longitudinal distance between the target rectangular area and the rectangular area adjacent to the search direction in the opposite direction belongs to the first longitudinal distance range or the second longitudinal distance range or the third longitudinal distance range, and the rectangular area adjacent to the search direction in the opposite direction is a target rectangular area or a rectangular area in the at least one rectangular area Q;
[0044] If the target rectangular area is found, the at least one rectangular area Q and the found target rectangular area are used as the at least one text area; or if the target rectangular area is not found, the at least one rectangular area Q is used as the at least one text area.
[0045] In this embodiment, Figure 4 The cloud application screen shown is used as an example to illustrate the text area recognition process. Figure 4 In the figure, assuming that area H and area I are the rectangular areas to which the recognized text belongs, the height of area H and the height of area I both belong to the first height range, and area H and area I satisfy the fifth positional relationship, then area H is used as a reference to search for the target rectangular area upward on the cloud application screen, and area I is used as a reference to search for the target rectangular area downward on the cloud application screen. Assuming that area H is used as a reference, the target rectangular area G is found upward on the cloud application screen, and area I is used as a reference to search for the target rectangular area J downward on the cloud application screen. Then the text area finally recognized includes the target rectangular area G, area H, area I and the target rectangular area J, and the text aggregation area covering the recognized text area can be Figure 4The area selected by the dotted box in the middle. Then, it is determined whether the text aggregation area is located at a specific position in the cloud application screen. If the text aggregation area is located at a specific position in the cloud application screen, it means that a pop-up window has appeared during the use of the cloud application by the corresponding user. For example, the absolute value of the difference between the ratio of the distance l5 from the text aggregation area to the left border of the cloud application screen and the width of the cloud application screen, and the ratio of the distance l6 from the text aggregation area to the right border of the cloud application screen and the width of the cloud application screen is less than a preset value, and the absolute value of the difference between the ratio of the distance h5 from the text aggregation area to the upper border of the cloud application screen and the height of the cloud application screen, and the ratio of the distance h6 from the text aggregation area to the lower border of the cloud application screen and the height of the cloud application screen is less than a preset value, it means that a pop-up window has appeared during the use of the cloud application by the corresponding user.
[0046] It should be noted that the fifth positional relationship between region H and region I can be that the longitudinal distance between region H and region I belongs to the first longitudinal distance range, that is, the line spacing between region H and region I is small. The process of searching for the target rectangular area upward / downward searches for a rectangular area on the cloud application screen whose height belongs to the first height range, the second height range, or the third height range, and whose longitudinal distance to the rectangular area adjacent to the search direction in the opposite direction belongs to the first longitudinal distance range, the second longitudinal distance range, or the third longitudinal distance range, that is, what is actually searched is a rectangular area with similar height and layout to region H and region I, as well as a title area and an operation component area. In other words, referring to the position and layout of the text in the cloud application screen in the pop-up window scene to be identified, the corresponding height conditions (i.e., the first height range, the second height range, and the third height range) and distance conditions (i.e., the first longitudinal distance range, the second longitudinal distance range, and the third longitudinal distance range) are designed. In the process of searching for the target rectangular area, if a rectangular area that meets the height condition and the distance condition is found, the rectangular area is used as a target rectangular area. The first height range is the height range corresponding to a line of text in the pop-up window, the second height range is the height range corresponding to the pop-up window title, and the third height range is the height range corresponding to the text on the operating component; the first vertical distance range is the distance range corresponding to the distance between two adjacent lines of text in the pop-up window (such as the text in area H and the text in area I), the second vertical distance range is the distance range corresponding to the distance between the title in the pop-up window and the first line of text, and the third vertical distance range is the distance range corresponding to the distance between the text on the operating component and the last line of text in the pop-up window. The first height range, the second height range, the third height range, the first vertical distance range, the second vertical distance range, and the third vertical distance range can be determined based on the position and layout of the text on the cloud application screen in the pop-up window scenario.
[0047] Based on the foregoing method embodiment, determining the text aggregation area according to the at least one text area may include:
[0048] An area covering the at least one text area is used as a text aggregation area.
[0049] Reference Figure 5 FIG. 1 is a schematic diagram of a structure of a scene recognition device for a cloud application provided in an embodiment of the present application, including:
[0050] The text recognition unit 50 is used to obtain a cloud application screen, perform text recognition on a designated area in the cloud application screen, and if text is recognized, perform text area recognition according to the position of the recognized text, wherein at least one text area is recognized when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition;
[0051] The scene recognition unit 51 is used to determine a text aggregation area according to the at least one text area if at least one text area is recognized, and perform target scene recognition on the cloud application according to the position of the text aggregation area in the cloud application screen.
[0052] The scene recognition device for cloud applications provided in the embodiment of the present application performs scene recognition based on the layout of text recognized from the cloud application screen. Compared with the existing technology of performing scene recognition by comparing the similarity between images, this solution can improve the efficiency of cloud application scene recognition.
[0053] Based on the above device embodiment, if the target scenario is authentication login, the text recognition unit can be used to:
[0054] If the position of the first type of characters and the position of the second type of characters satisfy a first position relationship, then both the area of the first type of characters and the area of the second type of characters are used as one of the at least one character area; or
[0055] If the position of the first category of characters and the position of the second category of characters satisfy a first position relationship, and the position of the first category of characters and the position of the third category of characters satisfy a second position relationship, then the area of the first category of characters, the area of the second category of characters and the area of the third category of characters are all used as one of the at least one character area, wherein the first position relationship is that the distance between the position of the first category of characters and the position of the second category of characters belongs to a first distance range, and the angle formed by the position of the first category of characters, the position of the second category of characters and the coordinate axis belongs to a first angle range, and the second position relationship is that the distance between the position of the first category of characters and the position of the third category of characters belongs to a second distance range, and the angle formed by the position of the first category of characters, the position of the third category of characters and the coordinate axis belongs to a second angle range.
[0056] Based on the above device embodiment, if the target scene is successfully entering the application, the text recognition unit can be used to:
[0057] If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, the area of the first group of numbers and the area of the second group of numbers are both used as one of the at least one text area; and / or
[0058] If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, and the position of the first group of numbers and the position of the third group of numbers satisfy a fourth position relationship, the area of the first group of numbers, the area of the second group of numbers and the area of the third group of numbers are all used as one of the at least one text area, wherein the third position relationship is that the distance between the position of the first group of numbers and the position of the second group of numbers belongs to a third distance range, and the angle formed by the position of the first group of numbers, the position of the second group of numbers and the coordinate axis belongs to a third angle range, and the fourth position relationship is that the distance between the position of the first group of numbers and the position of the third group of numbers belongs to a fourth distance range, and the angle formed by the position of the first group of numbers, the position of the third group of numbers and the coordinate axis belongs to a fourth angle range.
[0059] Based on the above device embodiment, if the target scene is a pop-up window, the text recognition unit may be used to:
[0060] Determine the rectangular area to which the recognized text belongs according to the position of the recognized text, and obtain at least one rectangular area Q;
[0061] If it is determined that the number of the at least one rectangular area Q is 1 and the height of the at least one rectangular area Q belongs to the first height range, or the number of the at least one rectangular area Q is not 1 and the height of the at least one rectangular area Q belongs to the first height range and any two adjacent rectangular areas in the at least one rectangular area Q satisfy the fifth positional relationship, at least one rectangular area in the at least one rectangular area Q is used as a reference, and a target rectangular area is searched upward and downward on the cloud application screen respectively, wherein the fifth positional relationship is that the longitudinal distance between the two rectangular areas belongs to the first longitudinal distance range, the target rectangular area is at least one rectangular area in the rectangular areas to which other texts except the recognized text belong in the cloud application screen, the height of the target rectangular area belongs to the first height range or the second height range or the third height range, the longitudinal distance between the target rectangular area and the rectangular area adjacent to the search direction in the opposite direction belongs to the first longitudinal distance range or the second longitudinal distance range or the third longitudinal distance range, and the rectangular area adjacent to the search direction in the opposite direction is a target rectangular area or a rectangular area in the at least one rectangular area Q;
[0062] If the target rectangular area is found, the at least one rectangular area Q and the found target rectangular area are used as the at least one text area; or if the target rectangular area is not found, the at least one rectangular area Q is used as the at least one text area.
[0063] Based on the above device embodiment, the scene recognition unit may be used for:
[0064] An area covering the at least one text area is used as a text aggregation area.
[0065] The scene recognition device for cloud applications provided in the embodiment of the present application has an implementation process that is consistent with the scene recognition method for cloud applications provided in the embodiment of the present application, and the effect that can be achieved is also the same as the scene recognition method for cloud applications provided in the embodiment of the present application, and will not be repeated here.
[0066] like Figure 6 As shown, an electronic device provided by an embodiment of the present application includes: a processor 60, a memory 61 and a bus 62, wherein the memory 61 stores machine-readable instructions executable by the processor 60, and when the electronic device is running, the processor 60 communicates with the memory 61 through the bus 62, and the processor 60 executes the machine-readable instructions to perform the steps of the scene recognition method of the cloud application as described above.
[0067] Specifically, the above-mentioned memory 61 and processor 60 can be general-purpose memories and processors, which are not specifically limited here. When the processor 60 runs the computer program stored in the memory 61, it can execute the scene recognition method of the above-mentioned cloud application.
[0068] Corresponding to the above-mentioned scene recognition method for cloud applications, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned scene recognition method for cloud applications are executed.
[0069] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A scene recognition method for cloud applications, characterized in that: include: Acquire a cloud application screen, perform text recognition on a designated area in the cloud application screen, and if text is recognized, perform text area recognition according to the position of the recognized text, wherein at least one text area is recognized when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition; If at least one text area is identified, a text aggregation area is determined based on the at least one text area, and target scene identification is performed on the cloud application based on the position of the text aggregation area in the cloud application screen. The text aggregation area needs to cover all identified text areas. After the text aggregation area is determined, whether a specific scene appears during the operation of the cloud application is determined based on the distance between the text aggregation area and the left and right boundaries and the upper and lower boundaries of the cloud application screen.
2. The method according to claim 1, characterized in that If the target scenario is authentication login, the text area recognition according to the position of the recognized text includes: If the position of the first type of characters and the position of the second type of characters satisfy a first position relationship, then both the area of the first type of characters and the area of the second type of characters are used as one of the at least one character area; or If the position of the first category of characters and the position of the second category of characters satisfy a first position relationship, and the position of the first category of characters and the position of the third category of characters satisfy a second position relationship, then the area of the first category of characters, the area of the second category of characters and the area of the third category of characters are all used as one of the at least one character area, wherein the first position relationship is that the distance between the position of the first category of characters and the position of the second category of characters belongs to a first distance range, and the angle formed by the position of the first category of characters, the position of the second category of characters and the coordinate axis belongs to a first angle range, and the second position relationship is that the distance between the position of the first category of characters and the position of the third category of characters belongs to a second distance range, and the angle formed by the position of the first category of characters, the position of the third category of characters and the coordinate axis belongs to a second angle range.
3. The method according to claim 1, characterized in that If the target scene is successfully entering the application, the text area recognition according to the position of the recognized text includes: If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, the area of the first group of numbers and the area of the second group of numbers are both used as one of the at least one text area; and / or If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, and the position of the first group of numbers and the position of the third group of numbers satisfy a fourth position relationship, the area of the first group of numbers, the area of the second group of numbers and the area of the third group of numbers are all used as one of the at least one text area, wherein the third position relationship is that the distance between the position of the first group of numbers and the position of the second group of numbers belongs to a third distance range, and the angle formed by the position of the first group of numbers, the position of the second group of numbers and the coordinate axis belongs to a third angle range, and the fourth position relationship is that the distance between the position of the first group of numbers and the position of the third group of numbers belongs to a fourth distance range, and the angle formed by the position of the first group of numbers, the position of the third group of numbers and the coordinate axis belongs to a fourth angle range.
4. The method according to claim 1, characterized in that If the target scene is a pop-up window, the text area recognition according to the position of the recognized text includes: Determine the rectangular area to which the recognized text belongs according to the position of the recognized text, and obtain at least one rectangular area Q; If it is determined that the number of the at least one rectangular area Q is 1 and the height of the at least one rectangular area Q belongs to the first height range, or the number of the at least one rectangular area Q is not 1 and the height of the at least one rectangular area Q belongs to the first height range and any two adjacent rectangular areas in the at least one rectangular area Q satisfy the fifth positional relationship, at least one rectangular area in the at least one rectangular area Q is used as a reference, and a target rectangular area is searched upward and downward on the cloud application screen respectively, wherein the fifth positional relationship is that the longitudinal distance between the two rectangular areas belongs to the first longitudinal distance range, the target rectangular area is at least one rectangular area in the rectangular areas to which other texts except the recognized text belong in the cloud application screen, the height of the target rectangular area belongs to the first height range or the second height range or the third height range, the longitudinal distance between the target rectangular area and the rectangular area adjacent to the search direction in the opposite direction belongs to the first longitudinal distance range or the second longitudinal distance range or the third longitudinal distance range, and the rectangular area adjacent to the search direction in the opposite direction is a target rectangular area or a rectangular area in the at least one rectangular area Q; If the target rectangular area is found, the at least one rectangular area Q and the found target rectangular area are used as the at least one text area; or if the target rectangular area is not found, the at least one rectangular area Q is used as the at least one text area.
5. A scene recognition device for cloud applications, characterized in that: include: a text recognition unit, used to obtain a cloud application screen, perform text recognition on a designated area in the cloud application screen, and if text is recognized, perform text area recognition according to the position of the recognized text, wherein at least one text area is recognized when the position of the recognized text satisfies a specific position relationship condition and / or the rectangular area to which the recognized text belongs satisfies a specific height condition; A scene recognition unit is used to determine a text aggregation area based on the at least one text area if at least one text area is recognized, and to perform target scene recognition on the cloud application based on the position of the text aggregation area in the cloud application screen. The text aggregation area needs to cover all recognized text areas. After the text aggregation area is determined, whether a specific scene appears during the operation of the cloud application is determined based on the distance between the text aggregation area and the left and right boundaries and the upper and lower boundaries of the cloud application screen.
6. The device according to claim 5, characterized in that If the target scenario is authentication login, the text recognition unit is used to: If the position of the first type of characters and the position of the second type of characters satisfy a first position relationship, the area of the first type of characters and the area of the second type of characters are both used as one of the at least one character area; or If the position of the first category of characters and the position of the second category of characters satisfy a first position relationship, and the position of the first category of characters and the position of the third category of characters satisfy a second position relationship, then the area of the first category of characters, the area of the second category of characters and the area of the third category of characters are all used as one of the at least one character area, wherein the first position relationship is that the distance between the position of the first category of characters and the position of the second category of characters belongs to a first distance range, and the angle formed by the position of the first category of characters, the position of the second category of characters and the coordinate axis belongs to a first angle range, and the second position relationship is that the distance between the position of the first category of characters and the position of the third category of characters belongs to a second distance range, and the angle formed by the position of the first category of characters, the position of the third category of characters and the coordinate axis belongs to a second angle range.
7. The device according to claim 5, characterized in that If the target scene is successfully entering the application, the text recognition unit is used to: If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, the area of the first group of numbers and the area of the second group of numbers are both used as one of the at least one text area; and / or If the position of the first group of numbers and the position of the second group of numbers satisfy a third position relationship, and the position of the first group of numbers and the position of the third group of numbers satisfy a fourth position relationship, the area of the first group of numbers, the area of the second group of numbers and the area of the third group of numbers are all used as one of the at least one text area, wherein the third position relationship is that the distance between the position of the first group of numbers and the position of the second group of numbers belongs to a third distance range, and the angle formed by the position of the first group of numbers, the position of the second group of numbers and the coordinate axis belongs to a third angle range, and the fourth position relationship is that the distance between the position of the first group of numbers and the position of the third group of numbers belongs to a fourth distance range, and the angle formed by the position of the first group of numbers, the position of the third group of numbers and the coordinate axis belongs to a fourth angle range.
8. The device according to claim 5, characterized in that If the target scene is a pop-up window, the text recognition unit is used to: Determine the rectangular area to which the recognized text belongs according to the position of the recognized text, and obtain at least one rectangular area Q; If it is determined that the number of the at least one rectangular area Q is 1 and the height of the at least one rectangular area Q belongs to the first height range, or the number of the at least one rectangular area Q is not 1 and the height of the at least one rectangular area Q belongs to the first height range and any two adjacent rectangular areas in the at least one rectangular area Q satisfy the fifth positional relationship, at least one rectangular area in the at least one rectangular area Q is used as a reference, and a target rectangular area is searched upward and downward on the cloud application screen respectively, wherein the fifth positional relationship is that the longitudinal distance between the two rectangular areas belongs to the first longitudinal distance range, the target rectangular area is at least one rectangular area in the rectangular areas to which other texts except the recognized text belong in the cloud application screen, the height of the target rectangular area belongs to the first height range or the second height range or the third height range, the longitudinal distance between the target rectangular area and the rectangular area adjacent to the search direction in the opposite direction belongs to the first longitudinal distance range or the second longitudinal distance range or the third longitudinal distance range, and the rectangular area adjacent to the search direction in the opposite direction is a target rectangular area or a rectangular area in the at least one rectangular area Q; If the target rectangular area is found, the at least one rectangular area Q and the found target rectangular area are used as the at least one text area; or if the target rectangular area is not found, the at least one rectangular area Q is used as the at least one text area.
