Spraying method, smart terminal and storage medium

Abstract

This application proposes a spraying method, a smart terminal, and a storage medium. The method includes: acquiring a preset operation; controlling a fluid material to spray in a virtual scene according to the preset operation; and outputting a sprayed image. Through the above technical solution, controlling the fluid material to spray according to the preset operation allows the fluid material to blend with a depth mesh, and makes the sprayed content in the image conform to the object surface, thereby effectively improving the spraying effect in AR scenes.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a spraying method, a smart terminal, and a storage medium. Background Technology

[0002] Augmented Reality (AR) technology is a technology that uses computer technology to apply virtual viewpoint information to the real world, allowing real environments and virtual objects to be superimposed on the same scene or space simultaneously. With technological advancements, AR technology, with its unique blend of virtual and real elements, is widely used in various fields, and drawing in AR spaces has become an important part of user demand.

[0003] In the process of conceiving and implementing this application, the inventors discovered at least the following problems: In some implementations, spraying is usually performed in AR space using a fixed AR camera depth distance. However, such spraying methods often have defects such as being unable to fit the surface of the object, being unable to dynamically adjust the spraying distance, and being unable to collide and interact with objects in the AR scene, thus resulting in poor spraying effects.

[0004] Therefore, it is necessary to propose a solution to improve the spraying effect in AR scenes.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides a spraying method, a smart terminal, and a storage medium, aiming to improve the spraying effect in AR scenes.

[0007] To address the aforementioned technical problems, this application provides a spraying method applicable to terminal devices (such as smart terminals), the spraying method comprising:

[0008] S10: Obtain preset operation;

[0009] S20: Control the fluid material to spray in the virtual scene according to the preset operation and output the spraying screen.

[0010] Optionally, the method may include the following steps before step S20:

[0011] Get the current frame;

[0012] Calculate or determine the depth information of at least one point in the current frame.

[0013] Optionally, the step of obtaining the current frame may be preceded by:

[0014] Determine whether the terminal device supports depth information calculation;

[0015] If the terminal device does not support depth information calculation, then determine or generate the first prompt message; and / or,

[0016] If the terminal device supports depth information calculation, then the step of obtaining the current frame is executed.

[0017] Optionally, after the step of calculating or determining the depth information of at least one frame point in the current frame, the method further includes:

[0018] Select a preset position point in the current frame;

[0019] The depth information of the image points is cached using preset position points as markers to obtain the depth information of the image frame.

[0020] Optionally, step S20 includes:

[0021] Spraying parameter information is determined or generated based on preset operations;

[0022] A depth grid is formed based on the depth information of the image frame points;

[0023] The fluid material is sprayed according to the spraying parameter information, so that the fluid material is fused with the depth grid to obtain spray points;

[0024] The spraying pattern is determined or generated based on the spraying points.

[0025] Optionally, the step of determining or generating the spraying image based on the spraying points includes:

[0026] Based on the depth information, the spray point is attached to the current frame to obtain the spray point information;

[0027] The spraying screen is determined or generated based on the spraying point information and the pre-stored anchor point information.

[0028] Optionally, the step of determining or generating the spraying image based on the spraying point information and pre-stored anchor point information further includes:

[0029] Feature points in the current frame are identified and stored as anchor point information.

[0030] Optionally, the method may include the following steps before step S10:

[0031] Identify whether the screen of the terminal device is in a click state;

[0032] If the terminal device's screen is in a tapped state, then obtain the preset operation; and / or,

[0033] If the screen of the terminal device is not in a click state, it is determined whether spraying is being performed in the virtual scene. If spraying is being performed, spraying is stopped.

[0034] This application also provides a smart terminal, including: a memory and a processor, wherein the memory stores a spraying program, and when the spraying program is executed by the processor, it implements the steps of any of the spraying methods described above.

[0035] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the spraying methods described above.

[0036] As described above, the spraying method of this application includes obtaining a preset operation; controlling a fluid material to spray in a virtual scene according to the preset operation; and outputting a sprayed image. Through the above technical solution, controlling the fluid material to spray according to the preset operation allows the fluid material to blend with the depth mesh, and makes the sprayed content in the sprayed image conform to the object surface, thereby effectively improving the spraying effect in AR scenes. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0038] Figure 1 A schematic diagram of the hardware structure of a smart terminal to implement the various embodiments of this application;

[0039] Figure 2 A communication network system architecture diagram provided for an embodiment of this application;

[0040] Figure 3 This is a schematic flowchart of the spraying method according to the first embodiment;

[0041] Figure 4 This is a schematic flowchart of the spraying method according to the second embodiment;

[0042] Figure 5 This is a schematic flowchart of the spraying method according to the third embodiment;

[0043] Figure 6 This is a schematic diagram illustrating the effect of spraying in an outdoor scene in the embodiments of this application;

[0044] Figure 7This is a schematic diagram illustrating the effect of spraying in an indoor scene in the embodiments of this application;

[0045] Figure 8 This is a schematic flowchart of the spraying method according to the fourth embodiment;

[0046] Figure 9 This is a schematic diagram of the spraying process in an embodiment of this application.

[0047] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0049] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0050] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, may be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used in this application, may be interpreted as inclusive, or mean any one or any combination thereof. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Similarly, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C." Exceptions to this definition only occur when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0051] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0052] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0053] It should be noted that step designations such as S10 and S20 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the protection scope of this application.

[0054] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0055] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0056] Smart terminals can be implemented in various forms. For example, the smart terminals described in this application may include smart terminals such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0057] The following description will use a smart terminal as an example. Those skilled in the art will understand that, apart from elements specifically designed for mobile purposes, the construction according to the embodiments of this application can also be applied to fixed-type terminals.

[0058] Please see Figure 1 This is a schematic diagram of the hardware structure of a smart terminal implementing various embodiments of this application. The smart terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an A / V (Audio / Video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111, etc. Those skilled in the art will understand that... Figure 1 The smart terminal structure shown does not constitute a limitation on the smart terminal. A smart terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0059] The following is combined with Figure 1 A detailed introduction to each component of the smart terminal:

[0060] The radio frequency unit 101 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 110; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. Furthermore, the radio frequency unit 101 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), and 5G, etc.

[0061] WiFi is a short-range wireless transmission technology. Smart terminals, through the WiFi module 102, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 1 WiFi module 102 is shown, but it is understood that it is not a necessary component of a smart terminal and can be omitted as needed without changing the essence of the invention.

[0062] The audio output unit 103 can convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into audio signals and output them as sound when the smart terminal 100 is in call signal receiving mode, call mode, recording mode, voice recognition mode, broadcast receiving mode, etc. Furthermore, the audio output unit 103 can also provide audio output related to specific functions performed by the smart terminal 100 (e.g., call signal receiving sound, message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0063] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 106. The image frames processed by the GPU 1041 can be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) in operating modes such as telephone call mode, recording mode, and voice recognition mode, and can process such sound into audio data. The processed audio (voice) data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 101 in telephone call mode. The microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to eliminate (or suppress) noise or interference generated during the reception and transmission of audio signals.

[0064] The smart terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the ambient light level, and the proximity sensor can turn off the display panel 1061 and / or backlight when the smart terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that can also be configured in the phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0065] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0066] User input unit 107 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the smart terminal. Optionally, user input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 1071), and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 110, and can receive and execute commands sent by processor 110. In addition, touch panel 1071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Optionally, other input devices 1072 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc., without being specifically limited here.

[0067] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Subsequently, the processor 110 provides corresponding visual output on the display panel 1061 based on the type of touch event. Although in Figure 1 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components to realize the input and output functions of the smart terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the smart terminal. The specific implementation is not limited here.

[0068] Interface unit 108 serves as an interface through which at least one external device can connect to smart terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 108 may be used to receive input (e.g., data, power, etc.) from the external device and transmit the received input to one or more elements within smart terminal 100, or it may be used to transmit data between smart terminal 100 and the external device.

[0069] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a program storage area and a data storage area. Optionally, the program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0070] The processor 110 is the control center of the smart terminal. It connects various parts of the smart terminal via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 109, and by calling data stored in the memory 109, it performs various functions and processes data of the smart terminal, thereby providing overall monitoring of the smart terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 110.

[0071] The smart terminal 100 may also include a power supply 111 (such as a battery) that supplies power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0072] although Figure 1 As not shown, the smart terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0073] To facilitate understanding of the embodiments of this application, the communication network system on which the smart terminal of this application is based is described below.

[0074] Please see Figure 2 , Figure 2 This application provides a communication network system architecture diagram. The communication network system is an LTE system based on the universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and the operator's IP services 204, which are connected in sequence.

[0075] Optionally, UE201 can be the aforementioned terminal 100, which will not be described in detail here.

[0076] E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. Optionally, eNodeB2021 can connect to other eNodeB2022 via backhaul (e.g., X2 interface), and eNodeB2021 connects to EPC203, providing access from UE201 to EPC203.

[0077] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gateway) 2034, PGW (Packet Data Network Gateway) 2035, and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME2031 is the control node that handles signaling between UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as the Home Location Register (not shown in the figure) and stores user-specific information such as service characteristics and data rates. All user data can be sent through SGW2034. PGW2035 can provide UE 201 IP address allocation and other functions. PCRF2036 is the policy and charging control decision point for service data flow and IP bearer resources. It selects and provides available policy and charging control decisions for the policy and charging enforcement function unit (not shown in the figure).

[0078] IP services 204 may include the Internet, intranet, IMS (IP Multimedia Subsystem), or other IP services.

[0079] Although the above description uses the LTE system as an example, those skilled in the art should understand that this application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems (such as 5G), etc., without limitation.

[0080] Based on the above-described intelligent terminal hardware structure and communication network system, various embodiments of this application are proposed.

[0081] First Embodiment

[0082] In the first embodiment of the spraying method, the spraying method of this application can use a smart terminal as the execution subject, and the smart terminal can be a smartphone as an example. (See reference...) Figure 3 , Figure 3 This is a schematic flowchart of a spraying method according to the first embodiment, the spraying method comprising the following steps:

[0083] S10, Obtain preset operation;

[0084] In this embodiment, after launching the AR application on a smartphone, the user can enable the AR application to perform preset operations through voice, gesture, and touch information. This embodiment does not impose specific limitations on these operations. To clearly illustrate the technical solution of this embodiment, this embodiment will use the example of a user clicking the screen to generate touch information for explanation.

[0085] Optionally, after obtaining the user's preset operation, the preset operation can be parsed to determine or generate information such as the material, color, and effect of the fluid material used for spraying, so as to determine or generate spraying parameter information for spraying. Different spraying parameters can achieve different spraying effects, such as simulating fog, smoke, water ink, and other spraying effects, thereby meeting different user needs.

[0086] S20: Control the fluid material to spray in the virtual scene according to the preset operation and output the spraying screen.

[0087] In this embodiment, optionally, the pre-calculated depth information refers to the depth information of each point in the current AR frame that has been determined or calculated. Spraying based on the depth information of each frame point can make the sprayed content blend and interact with objects in the AR scene, thereby producing a vivid spraying effect. In related technologies, a fixed depth distance of the AR camera is often used to spray and place objects in the AR space. Since the depth distance used in this method is fixed, the resulting sprayed content usually floats in the image and is separated from the AR scene space. Therefore, the spraying effect is difficult to meet the user's needs. In this application, the determined or calculated depth information is formed into a mesh. The sprayed content can collide with the formed mesh to produce splashing and other effects, making the AR effect more realistic.

[0088] Optionally, during the spraying process, depth information can be input into the relevant script as an input parameter, and fluid material can be sprayed out using a spray gun or spray canister. The vertex shader then blends the sprayed content with the depth mesh. Optionally, during the spraying process, relevant anchor point information, spray point information, and material information can be stored. By using the region depth information to fit the material into the AR space, a spraying effect that fits the curved surface can be formed, thus obtaining the sprayed image.

[0089] In this embodiment, a preset operation is obtained; according to the preset operation, a fluid material is controlled to be sprayed in a virtual scene, and a spraying image is output. Through the above technical solution, by controlling the spraying of the fluid material according to the preset operation, the fluid material can be fused with the depth mesh, and the spraying content in the spraying image can be made to fit the object surface, thereby effectively improving the spraying effect in the AR scene.

[0090] Second Embodiment

[0091] Reference Figure 4 , Figure 4 A schematic flowchart of a spraying method according to a second embodiment is shown. Based on the first embodiment of this application, the spraying method further includes the following steps:

[0092] Step S11: Obtain the current frame;

[0093] Optionally, after launching the AR application on the smart terminal, it is necessary to first determine whether the current device supports AR Depth functionality. If the current device supports AR Depth functionality, it can read AR image frames and then determine or calculate the depth information in the read image frames, specifically including:

[0094] Determine whether the terminal device supports depth information calculation;

[0095] If the terminal device does not support depth information calculation, then determine or generate the first prompt message; and / or,

[0096] If the terminal device supports depth information calculation, then the step of obtaining the current frame is executed.

[0097] Optionally, it can be determined whether the current terminal device supports depth information calculation, i.e. whether it supports AR Depth function. If the current device does not support it, a first prompt message is generated to prompt the user that the current device does not support the Depth function; and / or, if the current device supports it, AR screen frames can be read to obtain the current screen frame of the terminal device for depth information determination or calculation and spraying.

[0098] Step S12: Calculate or determine the depth information of at least one point in the current frame.

[0099] Optionally, in this embodiment, the depth information of all points in the current frame is calculated or determined. In other embodiments, some feature points may be selected to determine or calculate the depth information according to the actual situation. Furthermore, to improve system performance, this embodiment selects marker points for information storage, specifically including:

[0100] Select a preset position point in the current frame;

[0101] The depth information of the image points is cached using preset position points as markers to obtain the depth information of the image frame.

[0102] Optionally, in this embodiment, the focal aperture (default center of the screen) in the AR frame is selected as the preset position point, and the depth information of each frame point is cached using the preset position point as a marker. Since the spraying process is often a dynamic adjustment process, each frame can be cached using marker points, thereby enabling rapid retrieval of depth information and adjustment of the spraying effect during the spraying process. In this process, the user does not need to walk to the object location to spray, reducing difficulty and operation, making it more convenient and easier to use, and thus meeting the user's need to dynamically adjust the spraying distance.

[0103] This embodiment uses the above-described scheme to read AR image frames, determine or calculate the depth information of each point in the image frame and cache it, thereby enabling rapid retrieval of depth information during the spraying process. The depth information is then used for spraying to form a Mesh collision feedback effect, making the AR effect more realistic.

[0104] Third Embodiment

[0105] Based on the first and second embodiments of this application, this embodiment discloses a method for spraying fluid materials in a virtual scene according to preset operation control and outputting spraying images. Figure 5 This diagram illustrates a method for controlling fluid materials to be sprayed in a virtual scene according to preset operations and outputting a spraying image. Please refer to [link / reference]. Figure 5 The specific steps include:

[0106] Step S201: Determine or generate spraying parameter information according to preset operations;

[0107] Optionally, by parsing the preset operations, information such as the material, color, and effect of the fluid material used for spraying can be determined or generated, so as to determine or generate spraying parameter information for spraying. Different spraying parameters can produce different spraying effects, such as simulating fog, smoke, and water-ink spraying effects, thereby meeting different user needs.

[0108] Step S202: Form a depth grid based on the depth information of the image points in the image frame;

[0109] Optionally, after determining or calculating the depth information of the current frame, a depth grid can be formed based on the depth information of each point in the frame. The formed grid is equivalent to an object in AR space, and thus can be blended with the fluid material sprayed from the spray gun or spray can.

[0110] Step S203: Spray the fluid material according to the spraying parameter information, so that the fluid material merges with the depth mesh to obtain spraying points;

[0111] Optionally, after determining or generating information such as material, color, and effects, spraying parameter information can be obtained. The Depth information is then input into the relevant script, and the corresponding fluid material is sprayed out using a spray gun or spray can according to the spraying parameter information. The fluid material can simulate various complex spraying effects such as smoke, fog, ink, chalk, oil paint, and crayon. Optionally, after the fluid material is sprayed, it interacts and collides with the Depth mesh of similar spatial objects, thereby generating several spray points. (See reference...) Figure 6 and Figure 7 , Figure 6 This is a schematic diagram illustrating the effect of spraying in an outdoor scene according to an embodiment of this application. Figure 7 This is a schematic diagram of the spraying effect in an indoor scene in this application embodiment. As shown in the figure, the spraying method in this application embodiment is not limited to the application scene. Users can spray anytime and anywhere in different scenes in AR space. Users can use this method to write shaders based on the Depth area information to form atomized spraying effect or water-ink fluid effect.

[0112] Step S204: Determine or generate the spraying image based on the spraying points.

[0113] Optionally, after the sprayed material interacts and collides with the Depth mesh to generate spray points, a preliminary spraying effect is obtained. To further improve the realism of the spraying effect, the spray points can be aligned with the image frames, and the spraying image can be determined or generated by combining anchor point information. Specifically, this includes:

[0114] Based on the depth information, the spray point is attached to the current frame to obtain the spray point information;

[0115] The spraying screen is determined or generated based on the spraying point information and the pre-stored anchor point information.

[0116] Optionally, after the sprayed material interacts and collides with the Depth grid to generate spray points, the depth information corresponding to the screen points in each area of ​​the screen can be used to fit the objects in the AR space. In this way, users can place liquid paint or other materials on curved objects, and the materials will fit the curved objects more realistically.

[0117] Optionally, before determining or generating the spray painting image, relevant anchor point information can be stored to make the spray painting image more stable, specifically including:

[0118] Feature points in the current frame are identified and stored as anchor point information.

[0119] In this embodiment, it should be noted that the storage of anchor point information is not limited to before or after spraying. When the image quality of the frame is high, feature points can be selected from the image as anchor points and the anchor point information can be stored. After obtaining the spraying point information, the spraying point information can be added to the anchor points and stored to form a complete data structure, thereby making the obtained spraying image more stable.

[0120] This embodiment, through the above scheme, determines or generates spraying parameter information according to preset operations; forms a depth grid based on the depth information of the image frame points; sprays the fluid material according to the spraying parameter information, causing the fluid material to merge with the depth grid to obtain spraying points; and determines or generates the sprayed image based on the spraying points. By obtaining corresponding spraying effects according to different preset operations, different user needs can be met; the depth information allows the sprayed material to interact and collide with objects in the AR scene, while making the sprayed content fit more closely with spatial objects, resulting in a more stable image and thus improving the AR spraying effect.

[0121] Fourth embodiment

[0122] Reference Figure 8 , Figure 8 This is a schematic flowchart of the spraying method according to the fourth embodiment. Based on the first, second, and third embodiments of this application, the spraying method further includes the following steps:

[0123] Step S01: Identify whether the screen of the terminal device is in a click state;

[0124] Step S02: If the screen of the terminal device is in a click state, then obtain the preset operation; and / or,

[0125] Step S03: If the screen of the terminal device is not in a click state, determine whether to spray paint in the virtual scene;

[0126] Step S04: If spraying is in progress, stop spraying.

[0127] Reference Figure 9 , Figure 9 This is a schematic diagram of the spraying process in an embodiment of this application, as shown below. Figure 7 As shown, optionally, in this embodiment, when the AR application of the terminal device is running, it can be determined whether the user needs to spray by judging whether the screen is in a click state. If the screen of the terminal device is in a click state, it means that the user needs to spray, so the preset operation can be obtained to spray; if the screen is not in a click state, it means that the user does not need to spray at this time, and it is necessary to further judge whether spraying is in progress. If spraying is in progress, the spraying is stopped, thereby realizing the control of the spraying process and meeting the user's needs.

[0128] In this embodiment, the system identifies whether the screen of the terminal device is in a clicked state; if the screen is in a clicked state, a preset operation is obtained; if the screen is not in a clicked state, it determines whether spraying is being performed in the virtual scene; if spraying is being performed, spraying is stopped. This achieves effective control of the spraying process and meets different user spraying needs.

[0129] This application embodiment also provides a smart terminal, the smart terminal including: a memory, a processor, a communication bus, and / or a spraying program stored in the memory:

[0130] The communication bus is used to enable communication between the processor and the memory;

[0131] The processor is used to execute the spraying program to implement the steps of the above-described embodiments of the spraying method, which will not be repeated here.

[0132] This application also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the above-described spraying method embodiments.

[0133] The specific implementation of the storage medium in this application is basically the same as the embodiments of the spraying method described above, and will not be repeated here.

[0134] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the various possible implementations above.

[0135] This application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device with the chip installed performs the methods described in the various possible implementations above.

[0136] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0137] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0138] The units in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0139] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0140] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0141] The technical features of the present application can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0143] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, storage disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0144] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A spraying method, characterized in that, Includes the following steps: S10: Obtain preset operation; S20: Control the fluid material to be sprayed in the virtual scene according to the preset operation and output the spraying screen; Prior to S20, the following also includes: Get the current frame; Calculate or determine the depth information of at least one frame point in the current frame; S20 includes: Spraying parameter information is determined or generated based on preset operations; A depth grid is formed based on the depth information of the image frame points; The fluid material is sprayed according to the spraying parameter information, so that the fluid material is fused with the depth grid to obtain spray points; Based on the depth information, the spray point is attached to the current frame to obtain the spray point information; The spraying screen is determined or generated based on the spraying point information and the pre-stored anchor point information.

2. The method as described in claim 1, characterized in that, Before the step of acquiring the current frame, the following also includes: Determine whether the terminal device supports depth information calculation; If the terminal device does not support depth information calculation, then determine or generate the first prompt message; and / or, If the terminal device supports depth information calculation, then the step of obtaining the current frame is executed.

3. The method as described in claim 1, characterized in that, After the step of calculating or determining the depth information of at least one frame point in the current frame, the method further includes: Select a preset position point in the current frame; The depth information of the image points is cached using preset position points as markers to obtain the depth information of the image frame.

4. The method as described in claim 1, characterized in that, Before the step of determining or generating the spraying image based on the spraying point information and the pre-stored anchor point information, the method further includes: Feature points in the current frame are identified and stored as anchor point information.

5. The method according to any one of claims 1 to 3, characterized in that, The steps preceding step S10 also include: Identify whether the screen of the terminal device is in a click state; If the terminal device's screen is in a tapped state, then obtain the preset operation; and / or, If the screen of the terminal device is not in a click state, it is determined whether spraying is being performed in the virtual scene. If spraying is being performed, spraying is stopped.

6. A smart terminal, characterized in that, The smart terminal includes a memory and a processor, wherein the memory stores a spraying program, and when the spraying program is executed by the processor, it implements the steps of the spraying method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores a spraying program, which, when executed by a processor, implements the steps of the spraying method as described in any one of claims 1 to 5.

Images