Object recognition method and device, electronic equipment, storage medium and program product

By introducing special effects elements and dynamic display technology into augmented reality recognition technology, the problem of low utilization of graphics processing resources during the recognition process is solved and the user experience is improved.

CN119007054APending Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310553485.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In augmented reality recognition technology, relying solely on positioning prompts during the recognition process leads to low utilization of device graphics processing resources, resulting in boring user experience.

Method used

By introducing special effects elements during the recognition process, using target display methods, strokes, dynamic display and other technologies to display the recognition special effects, and controlling the recognition reference line movement and ripple diffusion through the chip shader to enhance the user's interactive experience.

Benefits of technology

It improves the utilization rate of graphics processing resources, enriches the recognition process, reduces the boring feeling of users waiting, and improves the user experience of augmented reality recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007054A_ABST
    Figure CN119007054A_ABST
Patent Text Reader

Abstract

The invention provides an object recognition method and device, electronic equipment, a computer readable storage medium and a computer program product, and is applied to the technical field of image processing and the field of map application. The method comprises the following steps: displaying an identification control for performing augmented reality identification on a to-be-identified object; in response to a trigger operation for the identification control, displaying an identification interface; when the to-be-recognized object exists in the recognition interface, executing augmented reality recognition operation on the to-be-recognized object, and displaying a recognition special effect corresponding to the to-be-recognized object in the process of executing the augmented reality recognition operation; and in response to the completion of the augmented reality recognition operation, displaying an augmented reality recognition result corresponding to the to-be-recognized object. Through the method and the device, the special effect elements can be added in the recognition process, the recognition process is enriched, and the utilization rate of graphic processing resources is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology and map application, and in particular to an object recognition method, device, electronic device, storage medium and program product. Background Art

[0002] Augmented reality (AR) recognition technology is a computer technology that combines virtual information with the real world to create an augmented reality experience. AR recognition technology uses cameras, sensors or other devices to capture or recognize images and sounds in the real world (collectively referred to as objects to be recognized), and combines this information with computer-generated virtual content to enhance the user's sensory experience.

[0003] In the related art, during the process of identifying an object to be identified, the object to be identified is simply provided with a location prompt to indicate that it is being identified, which results in a relatively boring identification process and a low utilization rate of graphics processing resources in the device. Summary of the invention

[0004] The embodiments of the present application provide an object recognition method, device, electronic device, computer-readable storage medium and computer program product, which can add special effect elements during the recognition process, enrich the recognition process, and improve the utilization rate of graphics processing resources.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides an object recognition method, including:

[0007] Displaying recognition controls for performing augmented reality recognition of an object to be recognized;

[0008] In response to a trigger operation on the identification control, displaying an identification interface;

[0009] When the object to be identified exists in the identification interface, an augmented reality identification operation is performed on the object to be identified, and during the process of performing the augmented reality identification operation, an identification special effect corresponding to the object to be identified is displayed;

[0010] In response to the augmented reality recognition operation being completed, an augmented reality recognition result corresponding to the object to be recognized is displayed.

[0011] The present application provides an object recognition device, including:

[0012] A first display module, used to display a recognition control for performing augmented reality recognition of an object to be recognized;

[0013] A second display module, configured to display an identification interface in response to a trigger operation on the identification control;

[0014] A third display module is used to perform an augmented reality recognition operation on the object to be recognized when the object to be recognized exists in the recognition interface, and to display a recognition effect corresponding to the object to be recognized during the process of performing the augmented reality recognition operation;

[0015] The fourth display module is used to display the augmented reality recognition result corresponding to the object to be recognized in response to the completion of the augmented reality recognition operation.

[0016] In the above scheme, the third display module is also used to display a specific area in the object to be identified using a target display method, and use the adopted target display method as the identification effect corresponding to the object to be identified; wherein, the target display method is used to indicate that augmented reality identification is being performed on the specific area, and the target display method includes at least one of the following display methods: display using preset colors, superimposed mask display, highlight display, stroke display, and dynamic display.

[0017] In the above scheme, the third display module is also used to display the effective recognition range and the recognition baseline within the effective recognition range in the recognition interface; when the entire area of ​​the object to be recognized is within the effective recognition range, an augmented reality recognition operation is performed on the object to be recognized; in the process of performing the augmented reality recognition operation, the recognition baseline is controlled to move on the object to be recognized; along with the movement of the recognition baseline, the recognition special effects corresponding to each sub-area in the object to be recognized are displayed, and the sub-areas are divided according to the distance between the sub-areas and the recognition baseline.

[0018] In the above scheme, the third display module is also used to display the recognition effects corresponding to each sub-area in the object to be identified using different display styles; wherein the different display styles are used to indicate that the recognition strength of each sub-area is different, and the recognition strength is negatively correlated with the distance between the corresponding sub-area and the recognition baseline.

[0019] In the above scheme, the third display module is also used to obtain a time displacement function for controlling the movement of the recognition baseline in a fragment shader that performs an augmented reality recognition operation, and control the recognition baseline to move on the object to be recognized according to the time displacement function; wherein the recognition baseline located at the current position in the object to be recognized is used to indicate that augmented reality recognition is being performed on the sub-area corresponding to the current position in the object to be recognized; the position of the recognition baseline on the object to be recognized is determined according to the time displacement function, and the fragment shader is controlled to perform special effects rendering processing on the target sub-area located at the position, so as to obtain and display the recognition special effects corresponding to the target sub-area.

[0020] In the above scheme, the device also includes: a fifth display module, which is used to perform an augmented reality recognition operation on the partial area of ​​the object to be recognized when the partial area of ​​the object to be recognized is located within the effective recognition range; in the process of performing the augmented reality recognition operation, the recognition special effects corresponding to the partial area are displayed, and the original content of other areas of the object to be recognized except the partial area is displayed; wherein, the recognition special effects include the original content of the partial area and the corresponding special effect elements.

[0021] In the above scheme, the third display module is also used to perform edge extraction processing on the collected image corresponding to the object to be identified through a fragment shader to obtain a corresponding edge image; superimpose the edge image with the collected image to obtain a corresponding stroke image, and display the stroke image as an identification special effect corresponding to the object to be identified.

[0022] In the above scheme, the third display module is also used to obtain a color value used to change the color of the edge image, and multiply the color value by the edge image in the fragment shader to obtain an edge image of a preset color; superimpose the edge image of the preset color with the captured image to obtain a stroke image of the target color, so as to display the stroke image of the target color as an identification special effect corresponding to the object to be identified.

[0023] In the above scheme, the third display module is also used to perform texture extraction on the captured image corresponding to the object to be identified through a fragment shader to obtain an image texture corresponding to the captured image; obtain an animation texture with varying ripple density, and fuse the animation texture with the image texture through the fragment shader to obtain a ripple image with varying ripple density; display the ripple image with varying ripple density as an identification special effect corresponding to the object to be identified; wherein the varying ripple density is used to indicate the identification progress of the augmented reality identification operation.

[0024] In the above scheme, the third display module is also used to obtain a ripple diffusion function for indicating that the ripple diffusion range changes with time; obtain ripple elements that fluctuate and diffuse from the ripple diffusion center to the surrounding areas according to the ripple diffusion function through a fragment shader; fuse the captured image corresponding to the object to be identified with the captured image through the fragment shader to obtain a ripple image with a changing ripple diffusion range, and use the ripple image with a changing ripple diffusion range as an identification special effect corresponding to the object to be identified; wherein the changing ripple diffusion range is used to indicate the identification progress of the augmented reality identification operation.

[0025] In the above scheme, the device also includes: a ripple element determination module, which is used to determine the offset parameters used to affect the coordinate offset of each pixel point in the acquired image according to the ripple diffusion function; obtain the pixel coordinate offset function, wherein the pixel coordinate offset function is used to dynamically change the coordinate offset of each pixel point in the acquired image; determine the final coordinates of each pixel point according to the original coordinates of each pixel point, the offset parameters and the pixel coordinate offset function; and perform ripple special effects rendering processing according to the ripple diffusion function according to the final coordinates of each pixel point to obtain the corresponding ripple element.

[0026] In the above scheme, the ripple element determination module is also used to multiply the original coordinates of each pixel point, the offset parameter and the pixel coordinate offset function to obtain the final coordinates of each pixel point when the ripple diffusion center is the center position of the acquired image; when the ripple diffusion center is other than the center position of the acquired image in the identification interface, obtain the coordinate difference between the original coordinates of each pixel point and the ripple diffusion center, and multiply the displacement difference of each pixel point, the offset parameter and the pixel coordinate offset function to obtain the final coordinates of each pixel point.

[0027] In the above scheme, the third display module is also used to perform augmented reality recognition operations on each object to be recognized when the number of objects to be recognized in the recognition interface is at least two, and in the process of performing the augmented reality recognition operation, different display styles are used to display the recognition effects corresponding to each object to be recognized; the fourth display module is also used to display the enhanced display recognition results corresponding to each object to be recognized.

[0028] In the above scheme, the third display module is also used to control the at least two objects to be identified in the identification interface to be in an optional state when the number of the objects to be identified is at least two; in response to the selection operation on the target identification object, perform an augmented reality identification operation on the target identification object, and display the identification effects corresponding to the target identification object during the execution of the augmented reality identification operation; the fourth display module is also used to display the enhanced display identification result corresponding to the target identification object.

[0029] In the above scheme, the third display module is also used to perform an augmented reality recognition operation on the target recognition object when the number of the objects to be recognized in the recognition interface is at least two, and to display the recognition effects corresponding to the target recognition object during the execution of the augmented reality recognition operation; wherein the recognition priority of the target recognition object is higher than the recognition priority of other objects to be recognized among the at least two objects to be recognized except the target recognition object; the fourth display module is also used to display the enhanced display recognition result corresponding to the target recognition object.

[0030] In the above scheme, the device also includes: a prediction module, which is used to determine the recognition priority of the target recognition object through the following processing: determining the influencing parameters used to affect the recognition priority of the target recognition object, and the influencing parameters include at least one of the following: the position of the target recognition object in the recognition interface, the object type to which the target recognition object belongs, and the historical interaction heat corresponding to the target recognition object; based on the influencing parameters, calling the neural network model for prediction processing to obtain the recognition priority of the target recognition object, wherein the neural network model is trained through the influencing parameters used to affect the recognition priority of the recognition object samples and the annotated recognition priority of the recognition object samples.

[0031] An embodiment of the present application provides an electronic device, including:

[0032] A memory for storing computer executable instructions or computer programs;

[0033] The processor is used to implement the object recognition method provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0034] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or a computer program for implementing the object recognition method provided in the embodiment of the present application when executed by a processor.

[0035] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the object recognition method provided by the embodiment of the present application is implemented.

[0036] The embodiments of the present application have the following beneficial effects:

[0037] By applying the embodiment of the present application, during the process of performing augmented reality recognition operation on the object to be recognized, the recognition special effects corresponding to the object to be recognized are displayed, that is, special effect elements are added to the recognition process, the recognition process is enriched, and the utilization rate of graphics processing resources is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of the architecture of an object recognition system 100 provided in an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application;

[0040] Figure 3 is a flow chart of an object recognition method provided in an embodiment of the present application;

[0041] Figure 4 is a display schematic diagram of the recognition special effect provided in an embodiment of the present application;

[0042] Figure 5 is a display schematic diagram of an identification interface provided in an embodiment of the present application;

[0043] Figure 6 is a schematic diagram of obtaining a stroked image provided in an embodiment of the present application;

[0044] Figure 7 is a schematic diagram of object recognition provided by an embodiment of the present application;

[0045] Figure 8 is a display schematic diagram of the recognition special effect provided in an embodiment of the present application;

[0046] Fig. 9 is a schematic diagram of a Sober operator provided in an embodiment of the present application;

[0047] Fig.10 Schematic diagram of spatial domain convolution operation provided by an embodiment of the present application;

[0048] Fig.11 is a schematic diagram of edge detection provided by an embodiment of the present application;

[0049] Fig.12 is a schematic diagram of displaying a stroked image provided in an embodiment of the present application;

[0050] Fig.13 It is a schematic diagram of wave stretching provided in an embodiment of the present application;

[0051] Fig.14 It is a schematic diagram of the pixel coordinate offset function provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0053] It is understandable that in the embodiments of the present application, related data such as user information (for example, when the object to be identified is the work of itself or the user, interactive data for the object to be identified, etc.) is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0054] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] In the following description, the terms "first\second..." are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0058] 1) Client: an application running in the terminal to provide various services, such as identification client, search client, instant messaging client, etc.

[0059] 2) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0060] The embodiments of the present application provide an object recognition method, device, electronic device, computer-readable storage medium, and computer program product, which can add special effect elements in the recognition process, enrich the recognition process, and improve the utilization rate of graphics processing resources. The exemplary application of the electronic device provided by the embodiments of the present application is described below. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart phones, smart speakers, smart watches, smart TVs, vehicle-mounted terminals, AR devices, etc., and can also be implemented as servers. Below, an exemplary application when the device is implemented as a terminal will be described.

[0061] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the object recognition system 100 provided in an embodiment of the present application. To support an exemplary application, the terminal (terminal 400-1 and terminal 400-2 are shown as examples) is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0062] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0063] In actual applications, the terminal is provided with clients, such as recognition clients, search clients, instant messaging clients, etc. These clients have augmented reality recognition (i.e., AR recognition) functions. When the user opens the client in the terminal for AR recognition, a recognition control for performing augmented reality recognition on the object to be recognized is displayed, and a recognition interface is displayed in response to a trigger operation on the recognition control; when there is an object to be recognized in the recognition interface, the terminal sends a captured image corresponding to the object to be recognized to the server 200; the server 200 performs an augmented reality (AR) recognition operation on the object to be recognized based on the captured image, and in the process of performing the augmented reality recognition operation, generates a corresponding recognition effect according to the captured image, and returns the recognition effect to the terminal for display; when the augmented reality recognition operation is completed, the server 200 obtains the augmented reality recognition result corresponding to the object to be recognized, and returns the augmented reality recognition result to the terminal for display.

[0064] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application, taking the electronic device 500 as Figure 1 For example, the terminal in Figure 2 The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 540 is not described in detail. Figure 2 Various buses are labeled as bus system 540 .

[0065] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiment of the present application is intended to include any suitable type of memory. The memory 550 optionally includes one or more storage devices that are physically far away from the processor 510.

[0067] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0068] The operating system 551 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic businesses and processing hardware-based tasks; a network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520. Exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.

[0069] In some embodiments, the object recognition device provided in the embodiments of the present application can be implemented in software. The object recognition device provided in the embodiments of the present application can be provided as various software embodiments, including various forms including applications, software, software modules, scripts or codes. Figure 2 An object recognition device 555 stored in a memory 550 is shown, which may be software in the form of a program and a plug-in, and includes a series of modules, including a first display module 5551, a second display module 5552, a third display module 5553 and a fourth display module 5554. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0070] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the object recognition method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0071] In some embodiments, the terminal or server can implement the object recognition method provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as instant messaging APP, identification APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0072] As mentioned above, the object recognition method provided in the embodiment of the present application can be implemented by various types of electronic devices, for example, Figure 1 The terminal in the command prompt can also be executed separately by Figure 1 The terminal and server 200 in the embodiment cooperate to execute. Figure 1 The object recognition method provided in the embodiment of the present application is described as an example in which the terminal alone executes the object recognition method provided in the embodiment of the present application. Figure 3 , Figure 3 is a flow chart of the object recognition method provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0073] Step 101: The terminal displays a recognition control for performing augmented reality recognition on an object to be recognized.

[0074] Here, in actual applications, a client with an augmented reality recognition function (i.e., an AR recognition function) is provided on the terminal, such as a search client or an instant messaging client with an AR recognition function, or a client with a small program with an AR recognition function embedded on the terminal, such as an instant messaging client provided on the terminal, and a small program with an AR recognition function embedded in the instant messaging client, such as a scenic spot guide small program. When a user opens a client or small program with an AR recognition function, a recognition control for AR recognition can be displayed, and the object to be recognized can be AR recognized through the recognition control.

[0075] Step 102: In response to a trigger operation on the identification control, display an identification interface.

[0076] In actual applications, when the user triggers the recognition control, the terminal responds to the trigger operation and displays a recognition interface for identifying the object to be identified. Through the recognition interface, a captured image corresponding to the object to be identified can be collected, and AR recognition of the object to be identified can be performed based on the captured image.

[0077] It should be noted that the trigger operation may be a click operation, a double-click operation, a sliding operation or other forms of operation, and the embodiments of the present application do not limit the specific form of the trigger operation.

[0078] Step 103: When there is an object to be identified in the identification interface, an augmented reality identification operation is performed on the object to be identified, and during the process of performing the augmented reality identification operation, an identification effect corresponding to the object to be identified is displayed.

[0079] In some embodiments, the terminal may display the recognition effects corresponding to the object to be recognized in the following manner: a specific area in the object to be recognized is displayed in a target display mode, and the adopted target display mode is used as the recognition effects corresponding to the object to be recognized; wherein the target display mode is used to indicate that augmented reality recognition is being performed on the specific area, and the target display mode includes at least one of the following display modes: display in a preset color, superimposed mask display, highlight display, stroke display, and dynamic display.

[0080] Here, when the object to be identified exists in the identification interface, the terminal can collect the captured image corresponding to the object to be identified through an image acquisition device (such as a camera, a camera, a sensor with an image acquisition function, etc.), so as to perform AR recognition on the object to be identified based on the captured image, and during the AR recognition process, display the corresponding recognition effect for the specific area of ​​the object to be identified that is being AR recognized (that is, being subjected to AR recognition operation, or being scanned), wherein the recognition effect is used to prompt the recognition progress of the object to be identified, such as using a target display style to display the specific area of ​​the object to be identified that is being AR recognized, so as to distinguish it from other areas of the object to be identified except the specific area (that is, the area not recognized by AR).

[0081] For example, a preset color (such as blue) is used to display a specific area, and a color different from the preset color (such as gray) is used to display other areas; or, a mask is superimposed on a specific area and not on other areas; or, a specific area is highlighted and other areas are displayed in grayscale; or, a specific area is displayed with a stroke and other areas are displayed conventionally (non-stroked); or, a specific area is displayed dynamically and other areas are displayed statically, etc. Of course, in actual applications, in order to further enrich the display style of the recognition process, the display style can be expanded by combining two or more target display styles according to the object attributes of the object to be recognized (such as scenic spot guide signs, animals, humans, plants, etc.), so as to obtain a richer display style to display a specific area, such as using a blue gradient stroke display style to display a specific area, and using a conventional display style (such as black and no stroke) to display other areas.

[0082] See also Figure 4 , Figure 4This is a display diagram of the recognition effects provided in an embodiment of the present application. Taking the object to be recognized as text as an example, when the object to be recognized 401 is located in the recognition interface, in the process of performing AR recognition operation on the object to be recognized 401, a blue gradient stroke font is used to display the partial text (i.e., a specific area) 402 of the object to be recognized that is being subjected to AR recognition operation, and a gray unstroked font is used to display the remaining text (i.e., other areas) 403 of the object to be recognized that is not currently being subjected to AR recognition operation, so as to prominently prompt the current recognition progress.

[0083] In some embodiments, when there is an object to be identified in the identification interface, the terminal can perform an augmented reality identification operation on the object to be identified in the following manner: display an effective identification range and an identification baseline within the effective identification range in the identification interface; when the entire area of ​​the object to be identified is within the effective identification range, perform an augmented reality identification operation on the object to be identified; accordingly, the terminal can display the identification effects corresponding to the object to be identified in the process of performing the augmented reality identification operation in the following manner: in the process of performing the augmented reality identification operation, control the identification baseline to move on the object to be identified; along with the movement of the identification baseline, display the identification effects corresponding to each sub-area in the object to be identified, wherein the sub-areas are divided according to the distance between the identification baselines.

[0084] See also Figure 5 , Figure 5 It is a display schematic diagram of the recognition interface provided by the embodiment of the present application. In the recognition interface (such as a mobile phone screen), an effective recognition range 501 and a recognition reference line 502 are displayed. When all areas of the object to be recognized are within the effective recognition range, an AR recognition operation is performed on the entire object to be recognized. In the process of performing the AR recognition operation, the recognition reference line 502 is controlled to move within the effective recognition range 501, thereby achieving a visual effect of controlling the movement of the recognition reference line on the object to be recognized. In the process of controlling the movement of the recognition reference line, the object to be recognized (or the corresponding collected image) can be divided into multiple sub-areas in real time according to the distance between each pixel point in the collected image corresponding to the object to be recognized and the recognition reference line, and the recognition special effects corresponding to each sub-area in the object to be recognized are displayed.

[0085] In some embodiments, the terminal can display the recognition effects corresponding to each sub-area in the object to be identified in the following manner: using different display styles to display the recognition effects corresponding to each sub-area in the object to be identified; wherein different display styles are used to indicate that the recognition strength of each sub-area is different, and the recognition strength is negatively correlated with the distance between the corresponding sub-area and the recognition baseline.

[0086] Here, before displaying the recognition effects corresponding to each sub-region, the recognition strength corresponding to each sub-region is first determined. The recognition strength depends on the absolute distance between the currently recognized sub-region and the recognition baseline. The recognition strength corresponding to each sub-region is determined according to the distance between each sub-region and the recognition baseline. For example, assuming that the position of the sub-region is y, the position of the recognition baseline is Sy, and the radius of the effective recognition range is r, the recognition strength I can be fitted using the following formula:

[0087] From the above formula, we can know that when r is known, the smaller the absolute distance between the sub-region and the recognition baseline, the greater the recognition strength corresponding to the sub-region, and the larger the distance between the sub-region and the recognition baseline, the smaller the recognition strength corresponding to the sub-region, such as Figure 5 As shown; then, for each sub-area, a display style corresponding to the recognition strength is adopted to display the recognition effect corresponding to the corresponding sub-area.

[0088] For example, when the display style of the recognition special effect is displayed in a stroke style, the closer the sub-area is to the recognition baseline, the stronger the corresponding recognition strength is, and the higher the corresponding superimposed stroke strength is, that is, for each sub-area from near to far away from the recognition baseline, the corresponding superimposed stroke strength is gradually weakened until it disappears (that is, no superimposed stroke); for another example, when the display style of the recognition special effect is displayed in an animation style, the closer the sub-area is to the recognition baseline, the stronger the corresponding recognition strength is, and the greater the corresponding action strength is, that is, for each sub-area from near to far away from the recognition baseline, the corresponding action strength is gradually weakened until it disappears (that is, it remains stationary), thereby achieving the animation effect of the gradual disappearance of the recognition special effect during the recognition process.

[0089] In some embodiments, the terminal can control the movement of the identification baseline on the object to be identified in the following manner: obtain a time displacement function for controlling the movement of the identification baseline, and control the movement of the identification baseline on the object to be identified according to the time displacement function through a fragment shader; wherein, the identification baseline located at the current position in the object to be identified is used to indicate that augmented reality identification is being performed on the sub-area corresponding to the current position in the object to be identified; accordingly, the terminal can display the identification special effects corresponding to each sub-area in the object to be identified in the following manner: determine the target position of the identification baseline on the object to be identified according to the time displacement function, and control the fragment shader to perform special effects rendering processing on the sub-area located at the target position, so as to obtain and display the identification special effects corresponding to the sub-area at the target position.

[0090] In practical applications, by adding a time displacement function to the fragment shader that performs augmented reality recognition operations, the position of the recognition baseline is controlled to change according to time, and the fragment shader is informed of which sub-areas need to be rendered with special effects at the current moment. Since the recognition baseline is used to indicate the progress of recognition, such as the recognition baseline located at the target position in the object to be recognized, it is used to indicate that AR recognition is being performed on the sub-area corresponding to the target position. Therefore, in the process of controlling the movement of the recognition baseline, the target position of the recognition baseline on the object to be recognized is determined in real time according to the time displacement function, and the sub-area corresponding to the target position is rendered with special effects, while the sub-areas corresponding to other positions except the target position are not rendered with special effects.

[0091] When performing special effects rendering processing on the sub-area corresponding to the target position, the fragment shader performs special effects rendering processing on the sub-area corresponding to the target position in the captured image after acquiring the captured image corresponding to the object to be identified (i.e., the initial camera image captured by the terminal), such as adding special effects elements to the sub-area corresponding to the target position, and superimposing the special effects elements with the captured image to obtain a superimposed image, so that the superimposed image is used as the camera image displayed by the terminal during the AR recognition process. At this time, the recognition special effect corresponding to the sub-area of ​​the target position is the special effects element in the superimposed image.

[0092] In some embodiments, when a partial area of ​​the object to be identified is within the effective identification range, an augmented reality recognition operation is performed on the partial area of ​​the object to be identified; during the process of performing the augmented reality recognition operation, the recognition special effects corresponding to the partial area are displayed, and the original content of other areas of the object to be identified except the partial area is displayed; wherein, the recognition special effects include the original content of the partial area and the corresponding special effect elements.

[0093] Here, when some areas of the object to be identified are within the effective identification range and some areas are outside the effective identification range, in the process of executing the augmented reality recognition operation, special effects rendering processing is performed on the part of the area of ​​the object to be identified that is within the effective identification range. For example, after the fragment shader acquires the captured image corresponding to the object to be identified (that is, the initial camera image captured by the terminal, that is, the original content), special effects rendering processing is performed on the part of the area in the captured image that is within the effective identification range, such as adding special effects elements to the original content of the part of the area within the effective identification range; while other areas of the object to be identified that are outside the effective identification range are not subjected to special effects rendering processing, and the original content of other areas is maintained; in the above manner, the recognition progress of the current AR recognition can be quickly judged through the displayed recognition special effects, thereby enriching the display style of the recognition process.

[0094] In some embodiments, the terminal can display the recognition effects corresponding to the object to be recognized in the following manner: perform image edge extraction processing on the captured image corresponding to the object to be recognized through a fragment shader to obtain a corresponding edge image; superimpose the edge image with the captured image to obtain a corresponding stroke image, and display the stroke image as the recognition effects corresponding to the object to be recognized.

[0095] Here, when the recognition effect is displayed in a stroke style, during the AR recognition process, the captured image is directly processed by the fragment shader to obtain the corresponding edge image (such as performing image edge extraction on the captured image through an edge extraction algorithm to obtain the horizontal edge image and vertical edge image corresponding to the captured image, and combining the horizontal edge image with the vertical edge image to obtain the edge image of the captured image), and the edge image is superimposed with the captured image to obtain a stroked image, and the stroked image is displayed as the recognition effect of the object to be recognized, so as to indicate that AR recognition is being performed on the object to be recognized through the stroked image.

[0096] In some embodiments, the terminal can perform texture blending processing on the edge image and the captured image to obtain a corresponding stroke image in the following manner: obtain a color value used to change the color of the edge image, and multiply the color value by the edge image in the fragment shader to obtain an edge image of a preset color; superimpose the edge image of the preset color with the captured image to obtain a stroke image of the target color, and display the stroke image of the target color as an identification effect corresponding to the object to be identified.

[0097] Here, after extracting the edge image from the captured image of the element, the color of the edge image can be adjusted and set in the fragment shader, such as multiplying the color value of the extracted edge image by the color value used to change the color of the edge image to obtain an edge image of a preset color. After setting the edge image to the preset color, the edge image of the preset color is superimposed with the original captured image to obtain a stroke image of the target color, and the stroke image of the target color is displayed as an identification special effect corresponding to the object to be identified. In this way, by customizing edge images of various colors, it is possible to easily obtain a stroke image of the desired target color, thereby improving convenience.

[0098] It should be noted that the preset color and the target color may be the same or different. When the acquired image is a grayscale image, the target color is the same as the preset color. When the acquired image is a color image, the target color is obtained by superimposing and mixing the preset color and the color of the acquired image. At this time, the target color and the preset color are usually different.

[0099] In some embodiments, the terminal can display the recognition effects corresponding to the object to be recognized in the following manner: perform texture extraction on the captured image corresponding to the object to be recognized through a fragment shader to obtain an image texture corresponding to the captured image; obtain an animation texture with varying ripple density, and fuse the animation texture with the image texture through a fragment shader to obtain a ripple image with varying ripple density; display the ripple image with varying ripple density as the recognition effects corresponding to the object to be recognized.

[0100] Among them, the changing ripple density is used to indicate the recognition progress of the augmented reality recognition operation. For example, before AR recognition, the ripple density is 0 (which can also be regarded as no ripples). As AR recognition proceeds, the ripple density starts from 0 and increases. The larger the ripple density (that is, the denser the ripples), the closer the AR recognition progress is to completion. In this way, during the recognition process, by displaying the recognition effect of a ripple image with changing ripple density, while enriching the display content during the recognition process, it is convenient for users to quickly understand the recognition progress, thereby improving the user experience of AR recognition.

[0101] In some embodiments, the terminal can display the recognition effects corresponding to the object to be recognized in the following manner: obtain a ripple diffusion function for indicating that the ripple diffusion range changes over time; obtain a ripple element that fluctuates and diffuses from the ripple diffusion center to the surrounding areas according to the ripple diffusion function through a fragment shader; fuse the captured image corresponding to the object to be recognized with the ripple element through a fragment shader to obtain a ripple image with a changing ripple diffusion range, and use the ripple image with a changing ripple diffusion range as the recognition effect corresponding to the object to be recognized; wherein the changing ripple diffusion range is used to indicate the recognition progress of the augmented reality recognition operation.

[0102] Here, the ripple diffusion function is used to indicate that the ripple diffusion range changes with time. When the ripple diffusion behavior is actually the radius of a circle that continues to increase with time, in the process of performing AR recognition on the object to be recognized, a ripple diffusion range with a change is used to indicate the AR recognition progress. For example, when the center of the collected image corresponding to the object to be recognized is used as the ripple diffusion center, the ripple elements that perform wave diffusion from the ripple diffusion center to the surrounding areas according to the ripple diffusion function, that is, the ripple elements that diffuse in a circular ring mode from the ripple diffusion center to the surrounding areas can be used to indicate that AR recognition is performed sequentially from the center of the collected image to the surrounding areas, that is, the image area in the collected image where AR recognition is being performed displays a corresponding recognition effect, and the recognition effect is a ripple image obtained by fusion and superposition of the original content of the image area and the corresponding ripple element. In this way, during the recognition process, by displaying the recognition effect of the ripple image with a changing ripple diffusion range, while enriching the display content during the recognition process, it is convenient for users to quickly understand the recognition progress, thereby improving the user experience of AR recognition.

[0103] In some embodiments, the terminal can obtain the ripple element in the following manner: determine the offset parameter used to affect the coordinate offset of each pixel point in the acquired image according to the ripple diffusion function; obtain the pixel coordinate offset function, wherein the pixel coordinate offset function is used to dynamically change the coordinate offset of each pixel point in the acquired image; determine the final coordinates of each pixel point according to the original coordinates, offset parameters and pixel coordinate offset function of each pixel point; and perform ripple special effects rendering processing according to the ripple diffusion function according to the final coordinates of each pixel point to obtain the corresponding ripple element.

[0104] Here, since the maximum range of ripple diffusion can be determined according to the ripple diffusion function, the maximum value of the coordinate offset of each pixel point should not exceed the maximum range of ripple diffusion, so the coordinate offset of each pixel point can be limited to a certain range through the ripple diffusion function. When calculating the offset parameter corresponding to each pixel point, the clamp (MIN, VAL, MAX) function can be used to obtain the corresponding offset parameter, wherein MIN (minimum value) and MAX (maximum value) can be set manually, MAX can be set to the maximum range of ripple diffusion, VAL is the coordinate offset of the pixel point, and the clamp (MIN, VAL, MAX) function is to limit VAL to a minimum value (MIN, which can be set, such as 0) and a maximum value (MAX, which can be set, such as 5). When VAL is less than MIN, the coordinate offset of the pixel point is set to MIN, when VAL is between the minimum value and the maximum value, the coordinate offset of the pixel point is set to VAL, and when VAL is greater than MAX, the coordinate offset of the pixel point is set to MAX.

[0105] The visual effect of a ripple image with a changing ripple diffusion range is implemented in a principle similar to stretching each pixel in the acquired image around the ripple diffusion center. The pixel coordinate offset function is used to dynamically change the coordinate offset of each pixel in the acquired image. The pixel coordinate offset function can be represented by a sine function that changes with time. That is, the pixel coordinate offset function can be used to dynamically change the coordinate offset of each pixel, so that the stretching strength of the current image changes periodically over time.

[0106] In this way, the final coordinates of each pixel can be determined according to the original coordinates, offset parameters and pixel coordinate offset function of each pixel, and then ripple special effects rendering is performed according to the ripple diffusion function according to the final coordinates of each pixel to obtain the corresponding ripple elements.

[0107] In some embodiments, the terminal can determine the final coordinates of each pixel point based on the original coordinates, offset parameters and pixel coordinate offset function of each pixel point in the following manner: when the ripple diffusion center is the center position of the captured image, multiply the original coordinates, offset parameters and pixel coordinate offset function of each pixel point to obtain the final coordinates of each pixel point; when the ripple diffusion center is a position other than the center position of the captured image in the recognition interface, obtain the coordinate difference between the original coordinates of each pixel point and the ripple diffusion center, and multiply the displacement difference, offset parameter and pixel coordinate offset function of each pixel point to obtain the final coordinates of each pixel point.

[0108] Here, when the ripple diffusion center is the center position of the acquired image (that is, the ripple diffusion center coincides with the center of the acquired image), the original coordinates, offset parameters and pixel coordinate offset function of each pixel are multiplied to obtain the final coordinates of each pixel, and then the ripple special effects rendering is performed according to the ripple diffusion function based on the final coordinates of each pixel to obtain the corresponding ripple element; when the ripple diffusion center is other than the center position of the acquired image in the recognition interface (that is, the ripple diffusion center does not coincide with the center of the acquired image), first, the coordinate difference between the original coordinates of each pixel and the ripple diffusion center is calculated, and then the coordinate difference corresponding to each pixel is multiplied with the offset parameter and the pixel coordinate offset function to obtain the final coordinates of each pixel, and then the ripple special effects rendering is performed according to the ripple diffusion function based on the final coordinates of each pixel to obtain the corresponding ripple element.

[0109] In some embodiments, the terminal may perform an augmented reality recognition operation on the object to be recognized in the following manner, and display the recognition effects corresponding to the object to be recognized during the process of performing the augmented reality recognition operation: when the number of objects to be recognized in the recognition interface is at least two, perform an augmented reality recognition operation on each object to be recognized respectively, and during the process of performing the augmented reality recognition operation, use different display styles to display the recognition effects corresponding to each object to be recognized; accordingly, the terminal may display the augmented reality recognition results corresponding to the object to be recognized in the following manner: display the enhanced display recognition results corresponding to each object to be recognized.

[0110] Here, when there are multiple objects to be identified in the identification interface, such as object 1 to be identified and object 2 to be identified, augmented reality recognition operations can be automatically performed on each object to be identified, such as performing AR recognition on object 1 to be identified, and performing AR recognition on object 1 to be identified. In the process of performing the augmented reality recognition operation, different display styles are used to display the recognition effects corresponding to each object to be identified, such as recognition effect 1 corresponding to object 1 to be identified, and recognition effect 2 corresponding to object 2 to be identified. When AR recognition is completed, the enhanced display recognition results corresponding to each object to be identified are displayed, such as enhanced display recognition result 1 corresponding to object 1 to be identified, and enhanced display recognition result 2 corresponding to object 2 to be identified. In this way, AR recognition can be performed on multiple objects to be identified in the identification interface, thereby improving recognition efficiency.

[0111] It should be noted that when performing AR recognition on multiple objects to be recognized, AR recognition operations for different objects to be recognized may be executed serially or in parallel, and the present application does not limit the execution order of AR recognition operations for different objects to be recognized.

[0112] In some embodiments, the terminal may perform an augmented reality recognition operation on the object to be recognized in the following manner, and display the recognition effects corresponding to the object to be recognized during the execution of the augmented reality recognition operation: when the number of the objects to be recognized in the recognition interface is at least two, control at least two objects to be recognized to be in an optional state; in response to a selection operation on the target recognition object, perform an augmented reality recognition operation on the target recognition object, and display the recognition effects corresponding to the target recognition object during the execution of the augmented reality recognition operation; accordingly, the terminal may display the augmented reality recognition result corresponding to the object to be recognized in the following manner: display the enhanced display recognition result corresponding to the target recognition object.

[0113] Here, when there are multiple objects to be identified in the identification interface, such as object 1 to be identified and object 2 to be identified, each object to be identified is controlled to be in a selectable state, and the user can select the target identification object (such as object 1 to be identified) for AR identification, and in the process of AR identification of the target identification object, the identification special effects corresponding to the target identification object are displayed, and after the AR identification operation for the target identification object is completed, the enhanced display identification result corresponding to the target identification object (i.e., object 1 to be identified) is displayed; in this way, the user can conveniently select the identification object to be identified, thereby improving the targetedness of AR identification.

[0114] In actual applications, when there are multiple objects to be identified in the identification interface, such as object 1 to be identified and object 2 to be identified, in order to facilitate distinction, different display styles can be used to display each object to be identified according to the identification priority corresponding to the object to be identified. For example, if the identification priority of object 1 to be identified is higher than that of object 2 to be identified, the first display style is used to display object 1 to be identified, and the second display style is used to display object 2 to be identified. The first display style is different from the second display style, such as different fonts, different text colors, different display backgrounds, different text transparencies, etc., so as to highlight object 1 to be identified and distinguish it from object 2 to be identified, so as to give the user a striking prompt and facilitate the user to quickly select the object to be identified with a higher recognition priority for AR recognition.

[0115] In some embodiments, the terminal may perform an augmented reality recognition operation on the object to be recognized in the following manner, and display the recognition effects corresponding to the object to be recognized during the execution of the augmented reality recognition operation: when the number of objects to be recognized in the recognition interface is at least two, perform an augmented reality recognition operation on the target object to be recognized, and display the recognition effects corresponding to the target object to be recognized during the execution of the augmented reality recognition operation; wherein the recognition priority of the target object to be recognized is higher than the recognition priority of other objects to be recognized among the at least two objects to be recognized except the target object to be recognized; accordingly, the terminal may perform an augmented reality recognition operation on the object to be recognized in the following manner, and display the recognition effects corresponding to the object to be recognized during the execution of the augmented reality recognition operation: display the enhanced display recognition result corresponding to the target object to be recognized.

[0116] Here, when there are multiple objects to be identified in the identification interface, such as object 1 to be identified and object 2 to be identified, usually, the identification priorities corresponding to different objects to be identified are different, that is, if the identification priority of object 1 to be identified is higher than the identification priority of object 2 to be identified, the terminal automatically selects the target identification object (that is, object 1 to be identified) with the highest identification priority from multiple objects to be identified for AR identification, and in the process of AR identification of the target identification object, displays the identification effects corresponding to the target identification object (that is, object 1 to be identified), and after completing the AR identification operation for the target identification object, displays the enhanced display identification result corresponding to the target identification object (that is, object 1 to be identified); in this way, the intelligence and pertinence of AR identification are improved.

[0117] In some embodiments, the terminal can determine the recognition priority of the target recognition object through the following processing: determine the influencing parameters used to influence the recognition priority of the target recognition object, wherein the influencing parameters include at least one of the following: the position of the target recognition object in the recognition interface, the object type to which the target recognition object belongs, and the historical interaction heat corresponding to the target recognition object; based on the influencing parameters, call the neural network model for predictive processing to obtain the recognition priority of the target recognition object, wherein the neural network model is trained through the influencing parameters used to influence the recognition priority of the recognition object samples and the annotated recognition priority of the recognition object samples.

[0118] Among them, the position of the target recognition object in the recognition interface, such as the middle position of the recognition interface, has a greater positive impact on the recognition priority of the target recognition object than the edge position; the object type to which the target recognition object belongs refers to the genus category corresponding to the target recognition object, including but not limited to: animals, plants, humans, text, display boards, etc., and different object types may correspond to different recognition priorities; the historical interaction heat corresponding to the target recognition object (such as historical access data, historical recognition data, historical interaction data, etc.); after obtaining the influencing parameters that affect the recognition priority of the target recognition object, the influencing parameters are input into the trained neural network model for prediction processing to obtain the recognition priority of the target recognition object, which can make the prediction more accurate. It can be understood that the above method can be used to predict the corresponding recognition priority for the object to be recognized.

[0119] It should be noted that before applying the neural network model, it is necessary to train the initial neural network model, and then put the trained neural network model into use, so as to realize the prediction of the recognition priority of the target recognition object through artificial intelligence technology in combination with the user's interaction with the target recognition object. Among them, the neural network model is obtained by training the influencing parameters of the recognition priority of the recognition object sample and the recognition priority of the recognition object sample annotation. For example, based on the recognition object sample, the historical interaction heat of the recognition object sample and the recognition priority of the recognition object sample annotation, the initial neural network model is called for prediction processing to obtain the predicted recognition priority. After determining the value of the loss function of the neural network model through the predicted recognition priority and the annotated recognition priority, it can be judged whether the value of the loss function exceeds the preset threshold. When the value of the loss function exceeds the preset threshold, the error signal of the neural network model is determined based on the loss function, and the error information is back-propagated in the neural network model, and the model parameters of each layer are updated during the propagation process. Among them, the embodiment of the present application does not limit the model structure of the neural network model. For example, the neural network model can be a convolutional neural network, a deep neural network, etc.; it is not limited to the form of the loss function, for example, it can be a cross entropy loss function, an L2 loss function, etc.

[0120] Here, back propagation is explained. The training sample data is input into the input layer of the neural network model, passes through the hidden layer, and finally reaches the output layer and outputs the result. This is the forward propagation process of the neural network model. Since there is an error between the output result of the neural network model and the actual result, the error between the output result and the actual value is calculated, and the error is back-propagated from the output layer to the hidden layer until it propagates to the input layer. In the process of back propagation, the value of the model parameter is adjusted according to the error, that is, the loss function is constructed according to the error between the output result and the actual value, and the partial derivatives of the loss function with respect to the model parameters are calculated layer by layer to generate the gradient of the loss function with respect to the model parameters of each layer. Since the direction of the gradient indicates the direction of error expansion, the gradient of the model parameters is inverted and summed with the original parameters of each layer model. The summed result is used as the updated model parameters of each layer, thereby reducing the error caused by the model parameters; the above process is continuously iterated until convergence.

[0121] Step 104: In response to the augmented reality recognition operation being completed, the augmented reality recognition result corresponding to the object to be recognized is displayed.

[0122] Here, when the augmented reality recognition operation performed on the object to be recognized is completed, the corresponding augmented reality recognition result is displayed. For example, when the AR model corresponding to the object to be recognized is recognized, the recognized AR model is displayed as the augmented reality recognition result corresponding to the object to be recognized; when the AR model corresponding to the object to be recognized is not recognized, the augmented reality recognition result indicating that the AR model corresponding to the object to be recognized is not recognized is displayed, or recommendation indication information indicating a recommendation to view AR models corresponding to other recognized objects is displayed, wherein the correlation between the other recognized objects and the object to be recognized is greater than the correlation threshold.

[0123] In actual applications, when the object recognition method provided by the embodiment of the present application is applied in a scenic area navigation application, if there are points of interest (POI) in the scenic area in the recognition interface, such as buildings, statues, lakes, etc. in the scenic area, an enhanced display recognition operation (i.e., AR recognition) can be performed on the points of interest, and during the AR recognition process, the recognition effects of the corresponding points of interest are displayed to indicate the recognition progress of the points of interest through the recognition effects, and when the AR recognition operation is completed, the AR recognition results corresponding to the points of interest are displayed.

[0124] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0125] See also Figure 6 , Figure 6It is a schematic diagram of obtaining a stroked image provided in an embodiment of the present application. In the related art, when a camera is used to scan and identify an object to be identified, after the camera acquires an image of the object to be identified (i.e., the camera screen), the camera extracts corresponding pixel information from the acquired image with the help of a canvas, and uses a series of image processing techniques (such as jsfeat) in an image database (such as a JavaScript computer vision library) to perform edge detection on the acquired image according to the pixel information to obtain a corresponding edge image; then, the edge image is loaded into a fragment shader and mixed with the original acquired image to obtain a corresponding stroked image, thereby achieving stroking of the camera screen and presenting a scanning effect.

[0126] However, the image stroke solution of the related technology, on the one hand, will take up a lot of computing resources in the process of converting into pixel information with the help of canvas and then using the image processing library for edge detection. In scenes such as cameras that require high-frequency real-time rendering and high frame rate image presentation, phenomena such as freezes, frame drops, and heat generation may occur; on the other hand, the accuracy of the extracted edge image is poor, and the final scanning effect experience is average.

[0127] To this end, this application directly implements the digital image processing logic for edge extraction in the fragment processor, performs image processing and drawing for each pixel (also called fragment), reduces the number of operation steps and the amount of code, and uses the high-performance image rendering capabilities of the Web Graphics Library (WebGL, Web Graphics Library) based on the graphics processor (GPU, Graphics Processing Unit), greatly reducing the computing overhead, no longer experiencing freezes, dropped frames, etc., and at the same time, the detected edges are more accurate, providing better picture effects. In addition, by using this method of directly rewriting the camera screen fragment processor and adding other texture or image processing logic, richer camera screen effects will be achieved.

[0128] See also Figure 7 , Figure 71 is a schematic diagram of object recognition provided by an embodiment of the present application. Taking a terminal provided with a client and a small program with an AR recognition function embedded in the client, such as a scenic spot guide small program as an example, the terminal displays a recognition interface for AR recognition in response to a trigger operation on a recognition control, and displays prompt information 702 in the recognition interface for prompting the user to align with a guide board pattern (i.e., the above-mentioned object to be recognized) 701. When the guide board pattern 701 is aligned based on the prompt information 702, the AR recognition operation can be performed on the guide board pattern 701. During the AR recognition operation, the recognition effect corresponding to the guide board pattern 701 is displayed. The recognition effect can be used to know the recognition progress of the guide sign pattern 701. For example, when the scanning animation moves to the text 703 above the guide sign pattern 701, a blue gradient stroke effect (i.e., the recognition effect) is displayed on the text 703 to indicate that AR recognition is being performed on the text 703, while the text 704 in the guide sign pattern 701 that has not been subjected to AR recognition operation does not display the recognition effect. In this way, during the recognition process, the technological effect of real-time stroke of the current camera screen is presented, reducing the boredom of the user during the waiting period and enhancing the user's experience of participating in AR interaction; when the recognition is successful, the AR model 705 corresponding to the guide sign pattern 701 is displayed.

[0129] In addition, you can also integrate new textures into the fragment shader and add new image processing logic to enrich the camera image, such as Figure 8 As shown, Figure 8 This is a display diagram of the recognition special effects provided in the embodiment of the present application, (a) is the water ripple effect, and the special effects of the water ripple effect are used as the recognition special effects, (b) is the water surface ripple effect, and the special effects of the water surface ripple effect are used as the recognition special effects; through the above method, during the AR recognition process, the user is given a visual effect as if he is observing the world on the sparkling water surface, bringing a richer experience.

[0130] Next, the implementation principles of various recognition effects are explained.

[0131] 1. Stroke effect

[0132] 1.1 Image edge extraction

[0133] During the process of AR recognition of the object to be recognized by the camera, edge detection is performed on the captured image corresponding to the object to be recognized (i.e., the current camera screen) through the fragment shader to obtain an edge image corresponding to the captured image. Since in the fragment shader of the camera screen, the color value of the fragment (corresponding to each pixel in the captured image) is generated one by one according to the corresponding uv texture coordinate sampling, the uv texture coordinates of the fragment are sampled in the fragment shader and its color value is stored.

[0134] Next, Fig. 9 ( Fig. 9 The 3*3 kernel Sober operator shown in the schematic diagram of the Sober operator provided in the embodiment of the present application is used as the convolution kernel template, and the sampled color value is used Fig.10 ( Fig.10 The spatial domain convolution operation method shown in the spatial domain convolution operation diagram provided in the embodiment of the present application performs a spatial domain convolution operation, that is, the color value obtained after the color value of the pixel located at the edge in the acquired image is transformed is larger, such as close to or equal to 1 (displayed as the white part in the edge image), and the color value obtained after the color value of the pixel located at the non-edge in the acquired image is transformed is smaller, such as close to or equal to 0 (displayed as the black part in the edge image). After the edge image is calculated, only the color value of the edge image is stored (that is, the color value obtained after the color value of the pixel in the acquired image is transformed). Since the gradient value of each pixel in the acquired image needs to be combined with the horizontal gradient value and the vertical gradient value of the pixel, taking the pixel I (x, y) as an example, assuming that the horizontal gradient of the pixel I (x, y) is Gx and the vertical gradient is Gy, the gradient intensity of the pixel I (x, y) is Direction After the gradient value of each pixel in the collected image is obtained, the gradient value of the collected image (ie, the edge image of the collected image) can be obtained.

[0135] See also Fig.11 , Fig.11 This is a schematic diagram of edge detection provided in an embodiment of the present application. After edge detection is performed on the captured image in (a), the horizontal edge image in (b) and the vertical edge image in (c) are obtained. Then, the horizontal edge image in (b) and the vertical edge image in (c) are fused to obtain the final edge image of the captured image, as shown in (d).

[0136] 1.2. Edge color change and overlay

[0137] After obtaining the edge image corresponding to the captured image, the color of the edge image can be modified. When modifying, the color value used to change the color of the edge image is obtained, and the color value used to change the color of the edge image is multiplied by the color value of the edge image in the fragment shader to obtain an edge image of a preset color; then the edge image of the preset color is superimposed with the original captured image to obtain a stroke image of the target color.

[0138] See also Fig.12 , Fig.12This is a display schematic diagram of a stroke image provided in an embodiment of the present application. After the color of the edge image (a) corresponding to the captured image is modified to obtain an edge image of a preset color (such as blue), it is superimposed with the original captured image (b) to obtain a stroke image (c) of the target color. In this way, by customizing edge images of various colors, a stroke image of the desired target color can be easily obtained, thereby improving convenience.

[0139] It should be noted that the preset color and the target color may be the same or different. When the acquired image is a grayscale image, the target color is the same as the preset color. When the acquired image is a color image, the target color is obtained by superimposing and mixing the preset color and the color of the acquired image. At this time, the target color and the preset color are usually different.

[0140] 1.3. Make scanning animation

[0141] like Figure 5 As shown, an effective recognition range 501 and a recognition reference line 502 are displayed in the recognition interface (such as a mobile phone screen), and the image area within the effective recognition range (i.e., the above-mentioned sub-area) participates in the stroke, and the image area outside the effective recognition range will remain the same. A time parameter (i.e., the above-mentioned time displacement function) is added to the fragment shader of the camera screen to control the position of the recognition reference line to change according to time, and to inform the fragment shader which fragments (i.e., the above-mentioned sub-area) need to be stroked at the current moment, thereby achieving a scanning animation effect.

[0142] In addition, the recognition strength parameter is added to the image area within the effective recognition range. The recognition strength depends on the absolute distance between the pixel point in the currently recognized image area and the recognition reference line. According to the distance between each pixel point and the recognition reference line, the recognition strength corresponding to each pixel point is determined. For example, assuming that the position of the pixel point is y, the position of the recognition reference line is Sy, and the radius of the effective recognition range is r, the recognition strength I can be fitted using the following formula: Therefore, it can be seen from the above formula that, when r is known, the smaller the absolute distance between the pixel point and the recognition baseline, the greater the recognition strength corresponding to the pixel point, and the higher the strength of the corresponding superimposed stroke, that is, for each pixel point from near to far away from the recognition baseline, the corresponding superimposed stroke strength gradually decreases until it disappears (that is, no superimposed stroke), thereby achieving the animation effect of the gradual disappearance of the recognition special effect during the recognition process.

[0143] In the above manner, the edge image in the camera picture is extracted in real time, and the edge image is dynamically superimposed with the original picture to achieve the streamer scanning animation effect.

[0144] 2. Water surface ripple effect

[0145] Use off-screen canvas to draw sequence frame images to obtain image data that can be read by the fragment shader, generate image texture, bind it to texture unit 7, input animation texture into the camera screen fragment shader, combine sequence frame animation texture with camera screen texture, and integrate to show the camera animation effect of water surface changes.

[0146] 3. Water wave effect

[0147] Here, we still rewrite the fragment shader to create an animation effect of water ripples for the camera screen. The principle of the visual effect of water surface ripple diffusion is similar to Fig.13 Each pixel in the collected image is stretched around the center of the fluctuation (i.e., the ripple diffusion center mentioned above), where: Fig.13 This is a schematic diagram of the wave stretching provided in an embodiment of the present application. In the fragment shader, the UV coordinates of each pixel point are recalculated, and the coordinate difference between the original UV coordinates of each pixel point (that is, the original coordinates mentioned above) and the center point of the fluctuation is used as the new UV coordinate of the corresponding pixel point, so as to achieve the purpose of sampling the offset coordinates of the original camera screen, that is, to achieve the stretching effect of all pixels on the screen around the center point of the fluctuation.

[0148] See also Fig.14 , Fig.14 This is a schematic diagram of the pixel coordinate offset function provided in an embodiment of the present application. A time parameter is added to the calculation of the new uv coordinate and the pixel coordinate offset function (sine function) is nested to dynamically change the offset of the uv coordinate, so that the current picture stretching strength changes periodically over time.

[0149] In addition, according to the law that water waves diffuse from the center of the wave, after diffusing from the center of the wave, the middle part that has already fluctuated and diffused will no longer fluctuate. The diffusion behavior of water waves is actually the radius of a circle that is constantly increasing. The ripples outside the circle are effective, and the ripples inside the circle are static. Therefore, an offset parameter that affects the coordinate offset of each pixel in the captured image is added to the fragment shader, and is updated in real time externally and passed into the fragment shader like the time parameter, representing the radius of the circular range of the current water wave diffusion. Each pixel point needs to calculate the distance between its current UV coordinate and the center point of the fluctuation to determine whether the pixel point is within the circular range, and use the clamp(MIN, VAL, MAX) function to obtain the offset parameter, where the clamp(MIN, VAL, MAX) function is to limit VAL to a minimum value (MIN, which can be set, such as 0) and a maximum value (MAX, which can be set, such as 5). When VAL exceeds the range of the minimum and maximum values, a preset value is selected between the minimum and maximum values ​​for use; that is, when the pixel point is within the circular range, its corresponding offset parameter is a preset value (such as VAL, which can be the distance difference between the UV coordinate of the pixel point and the center point of the fluctuation); when the pixel point is not within the circular range, its corresponding offset parameter is the maximum value.

[0150] The new UV coordinates of each pixel point obtained by calculating the coordinate difference between the original UV coordinates of each pixel point (i.e. the above-mentioned original coordinates) and the coordinate difference between the center point of the fluctuation, the offset parameters and the pixel coordinate offset function are multiplied to obtain the final coordinates of each pixel point. Then, ripple special effects rendering is performed according to the final coordinates of each pixel point to obtain water ripples that spread from the center of the fluctuation to the surrounding areas (i.e. the above-mentioned ripple elements). The captured image is fused with the water ripples through the fragment shader to obtain a ripple image with a changing water ripple diffusion range, and the ripple image is used as the recognition effect in the AR recognition process.

[0151] Through the above method, combined with the camera screen rendering mechanism provided by the AR recognition capability in the client, the AR recognition animation is innovatively produced directly in the fragment shader, such as dynamically adding a series of sequence frame transparent animation textures to the fragment shader that renders the camera screen to enrich the screen and create an AR environment atmosphere; in the fragment shader, the camera screen is added with algorithm logic such as stretching offset around the center point to produce the effects of water ripples and water surface ripples, making the camera screen in the AR recognition process novel and interesting, more interactive, and reducing the user's boredom while waiting for the AR recognition process. In addition, the captured image is processed and rendered with high performance at the GPU level, which greatly reduces the computing overhead, and no longer causes heating, freezing, frame drops, etc., providing a better user experience. At the same time, the embodiment of the present application customizes the image edge extraction algorithm in the fragment shader, provides better image edge processing effects, gets rid of the reference and dependence on other image processing libraries, and highlights its streamlined and efficient advantages in the tight memory usage environment of the mini program.

[0152] So far, the object recognition method provided by the embodiment of the present application has been described in combination with the exemplary application and implementation of the electronic device provided by the embodiment of the present application. The following will continue to describe the cooperation of various modules in the object recognition device 555 provided by the embodiment of the present application to implement the object recognition scheme.

[0153] The first display module 5551 is used to display an identification control for performing augmented reality identification on an object to be identified; the second display module 5552 is used to display an identification interface in response to a trigger operation on the identification control; the third display module 5553 is used to perform an augmented reality identification operation on the object to be identified when the object to be identified exists in the identification interface, and to display an identification effect corresponding to the object to be identified during the execution of the augmented reality identification operation; the fourth display module 5554 is used to display an augmented reality identification result corresponding to the object to be identified in response to the completion of the augmented reality identification operation.

[0154] In some embodiments, the third display module is also used to display a specific area in the object to be identified using a target display method, and use the adopted target display method as an identification effect corresponding to the object to be identified; wherein, the target display method is used to indicate that augmented reality identification is being performed on the specific area, and the target display method includes at least one of the following display methods: display using preset colors, superimposed mask display, highlight display, stroke display, and dynamic display.

[0155] In some embodiments, the third display module is also used to display an effective recognition range and an recognition baseline within the effective recognition range in the recognition interface; when the entire area of ​​the object to be recognized is within the effective recognition range, an augmented reality recognition operation is performed on the object to be recognized; in the process of performing the augmented reality recognition operation, the recognition baseline is controlled to move on the object to be recognized; along with the movement of the recognition baseline, recognition effects corresponding to each sub-area in the object to be recognized are displayed, and the sub-areas are divided according to the distance between them and the recognition baseline.

[0156] In some embodiments, the third display module is also used to display the recognition effects corresponding to each sub-area in the object to be identified using different display styles; wherein the different display styles are used to indicate that the recognition strength of each sub-area is different, and the recognition strength is negatively correlated with the distance between the corresponding sub-area and the recognition baseline.

[0157] In some embodiments, the third display module is also used to obtain a time displacement function for controlling the movement of the recognition baseline in a fragment shader that performs an augmented reality recognition operation, and control the recognition baseline to move on the object to be recognized according to the time displacement function; wherein the recognition baseline located at the current position in the object to be recognized is used to indicate that augmented reality recognition is being performed on the sub-region corresponding to the current position in the object to be recognized; the position of the recognition baseline on the object to be recognized is determined according to the time displacement function, and the fragment shader is controlled to perform special effects rendering processing on the target sub-region located at the position, so as to obtain and display the recognition special effects corresponding to the target sub-region.

[0158] In some embodiments, the device also includes: a fifth display module, which is used to perform an augmented reality recognition operation on a partial area of ​​the object to be recognized when the partial area of ​​the object to be recognized is within the effective recognition range; in the process of performing the augmented reality recognition operation, the recognition special effects corresponding to the partial area are displayed, and the original content of other areas of the object to be recognized except the partial area is displayed; wherein, the recognition special effects include the original content of the partial area and the corresponding special effect elements.

[0159] In some embodiments, the third display module is also used to perform edge extraction processing on the captured image corresponding to the object to be identified through a fragment shader to obtain a corresponding edge image; superimpose the edge image with the captured image to obtain a corresponding stroke image, and display the stroke image as an identification special effect corresponding to the object to be identified.

[0160] In some embodiments, the third display module is also used to obtain a color value used to change the color of the edge image, and multiply the color value by the edge image in the fragment shader to obtain an edge image of a preset color; superimpose the edge image of the preset color with the captured image to obtain a stroke image of the target color, so as to display the stroke image of the target color as an identification special effect corresponding to the object to be identified.

[0161] In some embodiments, the third display module is also used to perform texture extraction on the captured image corresponding to the object to be identified through a fragment shader to obtain an image texture corresponding to the captured image; obtain an animation texture with varying ripple density, and fuse the animation texture with the image texture through the fragment shader to obtain a ripple image with varying ripple density; display the ripple image with varying ripple density as an identification special effect corresponding to the object to be identified; wherein the varying ripple density is used to indicate the identification progress of the augmented reality identification operation.

[0162] In some embodiments, the third display module is also used to obtain a ripple diffusion function for indicating that the ripple diffusion range changes over time; obtain ripple elements that fluctuate and diffuse from the ripple diffusion center to the surrounding areas according to the ripple diffusion function through a fragment shader; fuse the captured image corresponding to the object to be identified with the captured image through the fragment shader to obtain a ripple image with a changing ripple diffusion range, and use the ripple image with a changing ripple diffusion range as an identification special effect corresponding to the object to be identified; wherein the changing ripple diffusion range is used to indicate the identification progress of the augmented reality identification operation.

[0163] In some embodiments, the device also includes: a ripple element determination module, which is used to determine, based on the ripple diffusion function, an offset parameter used to affect the coordinate offset of each pixel point in the acquired image; obtain a pixel coordinate offset function, wherein the pixel coordinate offset function is used to dynamically change the coordinate offset of each pixel point in the acquired image; determine the final coordinates of each pixel point based on the original coordinates of each pixel point, the offset parameter and the pixel coordinate offset function; and perform ripple special effects rendering processing according to the ripple diffusion function based on the final coordinates of each pixel point to obtain a corresponding ripple element.

[0164] In some embodiments, the ripple element determination module is also used to, when the ripple diffusion center is the center position of the acquired image, multiply the original coordinates of each of the pixel points, the offset parameter and the pixel coordinate offset function to obtain the final coordinates of each of the pixel points; when the ripple diffusion center is a position other than the center position of the acquired image in the identification interface, obtain the coordinate difference between the original coordinates of each of the pixel points and the ripple diffusion center, and multiply the displacement difference of each pixel point, the offset parameter and the pixel coordinate offset function to obtain the final coordinates of each pixel point.

[0165] In some embodiments, the third display module is also used to perform an augmented reality recognition operation on each object to be recognized when the number of objects to be recognized in the recognition interface is at least two, and in the process of performing the augmented reality recognition operation, use different display styles to display the recognition effects corresponding to each object to be recognized; the fourth display module is also used to display the enhanced display recognition results corresponding to each object to be recognized.

[0166] In some embodiments, the third display module is also used to control the at least two objects to be identified in the identification interface to be in an optional state when the number of the objects to be identified in the identification interface is at least two; in response to a selection operation on the target identification object, an augmented reality identification operation is performed on the target identification object, and in the process of performing the augmented reality identification operation, an identification effect corresponding to the target identification object is displayed; the fourth display module is also used to display the enhanced display identification result corresponding to the target identification object.

[0167] In some embodiments, the third display module is also used to perform an augmented reality recognition operation on the target recognition object when the number of the objects to be recognized in the recognition interface is at least two, and to display the recognition effects corresponding to the target recognition object during the process of performing the augmented reality recognition operation; wherein, the recognition priority of the target recognition object is higher than the recognition priority of other objects to be recognized among the at least two objects to be recognized except the target recognition object; the fourth display module is also used to display the enhanced display recognition result corresponding to the target recognition object.

[0168] In some embodiments, the device also includes: a prediction module, which is used to determine the recognition priority of the target recognition object through the following processing: determining the influencing parameters used to affect the recognition priority of the target recognition object, and the influencing parameters include at least one of the following: the position of the target recognition object in the recognition interface, the object type to which the target recognition object belongs, and the historical interaction heat corresponding to the target recognition object; based on the influencing parameters, calling a neural network model for prediction processing to obtain the recognition priority of the target recognition object, wherein the neural network model is trained by the influencing parameters used to affect the recognition priority of the recognition object samples and the annotated recognition priority of the recognition object samples.

[0169] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs the object recognition method described in the embodiment of the present application.

[0170] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the object recognition method provided by the present application embodiment, for example, Figure 3 The object recognition method is shown.

[0171] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0172] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0173] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0174] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0175] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for object recognition, It is characterized in that The method comprises: Displaying recognition controls for performing augmented reality recognition of an object to be recognized; In response to a trigger operation on the identification control, displaying an identification interface; When the object to be identified exists in the identification interface, an augmented reality identification operation is performed on the object to be identified, and during the process of performing the augmented reality identification operation, an identification special effect corresponding to the object to be identified is displayed; In response to the augmented reality recognition operation being completed, an augmented reality recognition result corresponding to the object to be recognized is displayed.

2. The method according to claim 1, It is characterized in that The displaying of the recognition effect corresponding to the object to be recognized includes: Displaying a specific area of ​​the object to be identified in a target display mode, and using the target display mode as an identification effect corresponding to the object to be identified; Among them, the target display mode is used to indicate that augmented reality recognition is being performed on the specific area, and the target display mode includes at least one of the following display modes: preset color display, superimposed mask display, highlight display, stroke display, and dynamic display.

3. The method according to claim 1, It is characterized in that When the object to be identified exists in the identification interface, performing an augmented reality identification operation on the object to be identified includes: Displaying an effective recognition range and a recognition reference line within the effective recognition range in the recognition interface; When the entire area of ​​the object to be identified is within the effective identification range, performing an augmented reality identification operation on the object to be identified; The displaying of the recognition effect corresponding to the object to be recognized during the process of performing the augmented reality recognition operation includes: During the process of performing the augmented reality recognition operation, controlling the recognition reference line to move on the object to be recognized; Along with the movement of the recognition reference line, the recognition effects corresponding to the sub-areas in the object to be recognized are displayed, and the sub-areas are divided according to the distances from the recognition reference line.

4. The method according to claim 3, It is characterized in that Displaying the recognition effects corresponding to each sub-region of the object to be recognized, including: Using different display styles to display the recognition effects corresponding to each sub-area of ​​the object to be recognized; The different display styles are used to indicate that the recognition strength of each sub-region is different, and the recognition strength is negatively correlated with the distance between the corresponding sub-region and the recognition reference line.

5. The method according to claim 3, It is characterized in that The controlling the identification reference line to move on the object to be identified includes: Acquire a time displacement function for controlling the movement of the identification reference line; Controlling the recognition reference line to move on the object to be recognized according to the time displacement function through a fragment shader; The identification reference line located at the current position in the object to be identified is used to indicate that augmented reality identification is being performed on the sub-region corresponding to the current position in the object to be identified; The displaying of the identification effects corresponding to the sub-regions of the object to be identified includes: The target position of the identification reference line on the object to be identified is determined according to the time displacement function, and the fragment shader is controlled to perform special effects rendering processing on the sub-region located at the target position to obtain and display the identification special effects corresponding to the sub-region of the target position.

6. The method according to claim 3, It is characterized in that The method further comprises: When a partial area of ​​the object to be identified is within the effective identification range, performing an augmented reality identification operation on the partial area of ​​the object to be identified; During the process of performing the augmented reality recognition operation, displaying the recognition special effects corresponding to the partial area, and displaying the original contents of other areas of the object to be recognized except the partial area; The recognition special effect includes the original content of the partial area and the corresponding special effect elements.

7. The method according to claim 1, It is characterized in that The displaying of the recognition effect corresponding to the object to be recognized includes: Performing edge extraction processing on the collected image corresponding to the object to be identified by a fragment shader to obtain a corresponding edge image; The edge image is superimposed with the collected image to obtain a corresponding stroked image, and the stroked image is displayed as a recognition special effect corresponding to the object to be recognized.

8. The method according to claim 7, It is characterized in that The step of performing texture blending processing on the edge image and the collected image to obtain a corresponding stroked image includes: Acquire a color value for changing the color of the edge image, and multiply the color value by the edge image in the fragment shader to obtain an edge image of a preset color; The edge image of the preset color is superimposed on the collected image to obtain a stroke image of the target color, so as to display the stroke image of the target color as a recognition special effect corresponding to the object to be recognized.

9. The method according to claim 1, It is characterized in that The displaying of the recognition effect corresponding to the object to be recognized includes: Extracting texture from the captured image corresponding to the object to be identified by using a fragment shader to obtain an image texture corresponding to the captured image; Acquire an animated texture with a varying ripple density, and fuse the animated texture with the image texture through the fragment shader to obtain a ripple image with a varying ripple density; Displaying the ripple image with varying ripple density as a recognition special effect corresponding to the object to be recognized; The changing ripple density is used to indicate the recognition progress of the augmented reality recognition operation.

10. The method according to claim 1, It is characterized in that The displaying of the recognition effect corresponding to the object to be recognized includes: Obtaining a ripple diffusion function indicating the variation of the ripple diffusion range over time; Obtaining, through a fragment shader, ripple elements that perform wave diffusion from a ripple diffusion center to surrounding areas according to the ripple diffusion function; The captured image corresponding to the object to be identified is merged with the ripple element by the fragment shader to obtain a ripple image with a change in the ripple diffusion range, and the ripple image with the change in the ripple diffusion range is used as an identification special effect corresponding to the object to be identified; The changing ripple diffusion range is used to indicate the recognition progress of the augmented reality recognition operation.

11. The method according to claim 10, It is characterized in that The method further comprises: Determining, according to the ripple diffusion function, an offset parameter for influencing a coordinate offset of each pixel point in the acquired image; Obtaining a pixel coordinate offset function, wherein the pixel coordinate offset function is used to dynamically change the coordinate offset of each pixel point in the acquired image; Determining the final coordinates of each pixel point according to the original coordinates of each pixel point, the offset parameter and the pixel coordinate offset function; According to the final coordinates of each pixel point, ripple special effects rendering processing is performed according to the ripple diffusion function to obtain a corresponding ripple element.

12. The method according to claim 11, It is characterized in that Determining the final coordinates of each pixel point according to the original coordinates of each pixel point, the offset parameter and the pixel coordinate offset function includes: When the ripple diffusion center is the center position of the collected image, the original coordinates of each pixel point, the offset parameter and the pixel coordinate offset function are multiplied to obtain the final coordinates of each pixel point; When the ripple diffusion center is a position other than the center position of the acquired image in the identification interface, the coordinate difference between the original coordinates of each pixel point and the ripple diffusion center is obtained, and the displacement difference, the offset parameter and the pixel coordinate offset function of each pixel point are multiplied to obtain the final coordinates of each pixel point.

13. The method of claim 1, It is characterized in that The performing of the augmented reality recognition operation on the object to be recognized, and displaying a recognition effect corresponding to the object to be recognized during the performing of the augmented reality recognition operation, includes: When the number of the objects to be identified in the identification interface is at least two, an augmented reality identification operation is performed on each of the objects to be identified, and in the process of performing the augmented reality identification operation, different display styles are used to display identification effects corresponding to each of the objects to be identified; The displaying of the augmented reality recognition result corresponding to the object to be recognized includes: The enhanced display recognition results corresponding to each of the objects to be recognized are displayed.

14. The method of claim 1, It is characterized in that The performing of the augmented reality recognition operation on the object to be recognized, and displaying a recognition effect corresponding to the object to be recognized during the performing of the augmented reality recognition operation, includes: When the number of the objects to be identified in the identification interface is at least two, controlling the at least two objects to be identified to be in a selectable state; In response to a selection operation on a target recognition object, performing an augmented reality recognition operation on the target recognition object, and displaying a recognition effect corresponding to the target recognition object during the process of performing the augmented reality recognition operation; The displaying of the augmented reality recognition result corresponding to the object to be recognized includes: Display the enhanced display recognition result corresponding to the target recognition object.

15. The method of claim 1, It is characterized in that The performing of the augmented reality recognition operation on the object to be recognized, and displaying a recognition effect corresponding to the object to be recognized during the performing of the augmented reality recognition operation, includes: When the number of the objects to be identified in the identification interface is at least two, an augmented reality identification operation is performed on the target identification object, and during the process of performing the augmented reality identification operation, an identification special effect corresponding to the target identification object is displayed; The recognition priority of the target recognition object is higher than the recognition priority of other objects to be recognized among the at least two objects to be recognized except the target recognition object; The displaying of the augmented reality recognition result corresponding to the object to be recognized includes: Display the enhanced display recognition result corresponding to the target recognition object.

16. The method of claim 15, It is characterized in that The method further comprises: The recognition priority of the target recognition object is determined by the following processing: Determine an influencing parameter for influencing the recognition priority of the target recognition object, wherein the influencing parameter includes at least one of the following: the position of the target recognition object in the recognition interface, the object type to which the target recognition object belongs, and the historical interaction heat corresponding to the target recognition object; Based on the influencing parameters, a neural network model is called to perform prediction processing to obtain the recognition priority of the target recognition object, wherein the neural network model is trained by influencing parameters for influencing the recognition priority of recognition object samples and the annotated recognition priority of the recognition object samples.

17. An object recognition device, It is characterized in that The device comprises: A first display module, used to display a recognition control for performing augmented reality recognition of an object to be recognized; A second display module, configured to display an identification interface in response to a trigger operation on the identification control; A third display module is used to perform an augmented reality recognition operation on the object to be recognized when the object to be recognized exists in the recognition interface, and to display a recognition effect corresponding to the object to be recognized during the process of performing the augmented reality recognition operation; The fourth display module is used to display the augmented reality recognition result corresponding to the object to be recognized in response to the completion of the augmented reality recognition operation.

18. An electronic device, It is characterized in that include: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the object recognition method according to any one of claims 1 to 16 when executing the computer executable instructions or computer program stored in the memory.

19. A computer-readable storage medium, It is characterized in that Computer executable instructions or computer programs are stored, and when the computer executable instructions or computer programs are executed by a processor, the object recognition method according to any one of claims 1 to 16 is implemented.

20. A computer program product comprising a computer program or computer executable instructions, It is characterized in that When the computer program or computer executable instructions are executed by a processor, the object recognition method according to any one of claims 1 to 16 is implemented.