Data processing method and apparatus

Abstract

This specification provides a data processing method and apparatus. One data processing method includes: performing virtual object recognition on the descriptive text of a user's voice description segment in a sleep state uploaded by a service application to obtain virtual objects; constructing visualized scene data based on the virtual objects; performing emotional intent analysis on the virtual objects to obtain the user's emotional intent; constructing a group interaction task based on the user's emotional intent to obtain a user group scene interaction task; and after detecting an interaction instruction in the visualized scene data, performing task interaction processing on the group interaction task based on the visualized scene data. This process transforms the user's voice description segment in a sleep state into a visualized scene and uses the resulting visualized scene data to achieve group interaction.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a data processing method and apparatus. Background Technology

[0002] With the continuous promotion of internet technology and artificial intelligence, the development of online services based on the internet is becoming increasingly rapid, enabling users to meet diverse service needs through online services. In this process, visualization technology provides a feasible channel for the implementation of online services, providing users with intuitive and efficient ways to present information. However, the increasing number of online service providers and the increasing demands of users for online services have brought certain pressures and challenges to these providers. Summary of the Invention

[0003] This specification provides one or more embodiments of a data processing method, comprising: acquiring descriptive text of a user's voice description segment in a sleep state uploaded by a service application; performing virtual object recognition on the descriptive text to obtain virtual objects, and constructing a visualization scene based on the virtual objects to obtain visualization scene data; performing emotional intent parsing based on the virtual objects to obtain the user's emotional intent, and constructing a group interaction task for a user group based on the user's emotional intent; and if an interaction instruction from the visualization scene data is detected, performing task interaction processing for the group interaction task based on the visualization scene data.

[0004] This specification provides one or more embodiments of another data processing method, including: converting a user's voice description segment while asleep into speech, and uploading the resulting descriptive text to an application server. The method also involves rendering and displaying visualized scene data issued by the application server, and uploading the user's interaction commands to the visualized scene data to the application server, thereby performing task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent for a user group, and the user's emotional intent is obtained through emotional intent recognition based on virtual objects identified from the descriptive text.

[0005] This specification provides one or more embodiments of a data processing apparatus, including: a text acquisition module configured to acquire descriptive text of a user's voice description segment in a sleep state uploaded by a service application; an object recognition module configured to perform virtual object recognition on the descriptive text to obtain virtual objects, and construct a visualization scene based on the virtual objects to obtain visualization scene data; an intent parsing module configured to perform emotional intent parsing based on the virtual objects to obtain the user's emotional intent, and construct a group interaction task for a user group based on the user's emotional intent; and a task interaction module configured to perform task interaction processing of the group interaction task based on the visualization scene data if an interaction instruction is detected in the visualization scene data.

[0006] This specification provides one or more embodiments of another data processing apparatus, including: a speech conversion module configured to convert a user's speech description segment while asleep into speech, and upload the speech-converted description text to an application server; and an instruction upload module configured to render and display visualized scene data issued by the application server, and upload the user's interaction instructions on the visualized scene data to the application server, for task interaction processing of group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent for a user group, and the user's emotional intent is obtained by identifying emotional intent from virtual objects recognized from the description text.

[0007] This specification provides one or more embodiments of a data processing device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: acquire descriptive text of a user's voice description segment in a sleep state uploaded by a service application; perform virtual object recognition on the descriptive text to obtain virtual objects, and construct a visualization scene based on the virtual objects to obtain visualization scene data; perform emotional intent parsing based on the virtual objects to obtain user emotional intent, and construct a group interaction task for a user group based on the user emotional intent; if an interaction instruction from the visualization scene data is detected, perform task interaction processing for the group interaction task based on the visualization scene data.

[0008] This specification provides one or more embodiments of another data processing device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: convert a user's speech description segment while in a sleep state into speech, and upload the speech-converted description text to an application server; render and display visual scene data issued by the application server, and upload the user's interaction instructions on the visual scene data to the application server, so as to perform task interaction processing for group interaction tasks based on the visual scene data. The group interaction task is constructed based on the user's emotional intent for a user group, and the user's emotional intent is obtained by identifying emotional intent from virtual objects identified from the description text.

[0009] This specification provides one or more embodiments of a computer-readable storage medium for storing computer-executable instructions, which, when executed, perform the following steps: acquiring descriptive text of a user's voice description segment in a sleep state uploaded by a service application; performing virtual object recognition on the descriptive text to obtain virtual objects, and constructing a visualization scene based on the virtual objects to obtain visualization scene data; performing emotional intent parsing based on the virtual objects to obtain the user's emotional intent, and constructing a group interaction task for a user group based on the user's emotional intent; if an interaction instruction from the visualization scene data is detected, performing task interaction processing for the group interaction task based on the visualization scene data.

[0010] This specification provides one or more embodiments of another computer-readable storage medium for storing computer-executable instructions that, when executed, perform the following steps: converting a user's voice description segment while asleep into speech, and uploading the resulting description text to an application server; rendering and displaying visual scene data issued by the application server, and uploading the user's interaction instructions on the visual scene data to the application server, thereby performing task interaction processing for a group interaction task based on the visual scene data. The group interaction task is constructed based on the user's emotional intent for a user group, and the user's emotional intent is obtained through emotional intent recognition based on virtual objects identified from the description text. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 A schematic diagram illustrating the implementation environment of a data processing method provided in one or more embodiments of this specification; Figure 2 A data processing method flowchart provided for one or more embodiments of this specification; Figure 3 A schematic diagram of a voice acquisition page provided for one or more embodiments of this specification; Figure 4 A schematic diagram of a voice input page provided for one or more embodiments of this specification; Figure 5 This is a schematic diagram illustrating a scene generation page provided for one or more embodiments of this specification; Figure 6 A schematic diagram of a report display page provided for one or more embodiments of this specification; Figure 7 A schematic diagram of a scene sharing page provided for one or more embodiments of this specification; Figure 8 This is a schematic diagram of a group interaction page provided for one or more embodiments of this specification; Figure 9 A timing diagram illustrating a data processing method for a home interaction scenario provided in one or more embodiments of this specification; Figure 10 A flowchart illustrating another data processing method provided in one or more embodiments of this specification; Figure 11 A schematic diagram of an embodiment of a data processing apparatus provided in one or more embodiments of this specification; Figure 12 A schematic diagram of another embodiment of a data processing apparatus provided in one or more embodiments of this specification; Figure 13 This specification provides a schematic diagram of the structure of a data processing device according to one or more embodiments. Figure 14 This is a schematic diagram of the structure of another data processing device provided in one or more embodiments of this specification. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] The data processing methods provided in one or more embodiments of this specification are applicable to the implementation environment of visual scene interaction. (Refer to...) Figure 1 The implementation environment includes at least: Application server 101, service application 102; The application server 101 is used to perform virtual object recognition on the description text of the user's voice description segment in sleep state uploaded by the service application to obtain virtual objects, construct visualized scene data based on the virtual objects, and perform emotional intent analysis based on the virtual objects to obtain the user's emotional intent. Based on the user's emotional intent, it constructs group tasks to obtain group interaction tasks. After detecting the interaction instructions of the visualized scene data, it performs task interaction processing of the group interaction task based on the visualized scene data. The application server 101 can be deployed on a server, which can be one or more servers, a server cluster composed of several servers, or a cloud server of a cloud computing platform. Service application 102 is used to upload the descriptive text of the user's voice description segment in the sleep state to the application server, render and display the visualized scene data issued by the application server, and upload the user's interaction instructions on the visualized scene data to the application server. Service application 102 can be deployed on the user terminal, which can be a mobile phone, personal computer, tablet computer, e-book reader, wearable device, device for information interaction based on AR (Augmented Reality) / VR (Virtual Reality), and laptop computer, etc. In this implementation environment, application server 101 performs virtual object recognition on the descriptive text of the user's voice description segment in sleep state uploaded by service application 102 to obtain virtual objects. Based on the virtual objects, it constructs visualized scene data and performs emotional intent parsing based on the virtual objects to obtain the user's emotional intent. Based on the user's emotional intent, it constructs scene interaction tasks for user groups to obtain group interaction tasks. After detecting the interaction instructions of the visualized scene data, it performs task interaction processing of the group interaction task based on the visualized scene data. In this way, it performs visualized scene transformation on the user's voice description segment in sleep state and realizes group interaction through the visualized scene data obtained by the transformation.

[0014] One or more embodiments of a data processing method provided in this specification are as follows: Reference Figure 2 The data processing method provided in this embodiment can be applied to the application server, specifically including steps S202 to S208.

[0015] Step S202: Obtain the description text of the user's voice description segment in sleep state uploaded by the service application.

[0016] The service application described in this embodiment can be an application or a subroutine, specifically a payment application or a subroutine within a payment application; the service application can be in a specific user service state or a specific user mode, such as a service application in a youth mode.

[0017] The voice description segment may include sleep talking segments of the user while asleep or a scene description of the user's brain activity while asleep; here, the scene of the user's brain activity while asleep can be a dream scene, and the voice description segment can be input by the user while asleep or while awake. That is, the user can input sleep talking segments while asleep, and the user can input scene descriptions of the user's brain activity while asleep while awake.

[0018] In practice, the service application can convert the user's voice description segment in the sleep state into speech and upload the resulting description text to the application server. In this step, the service application can obtain the description text of the user's voice description segment in the sleep state uploaded by the service application. The user can be a specific user, such as a teenager, a child, an elderly user, or a pregnant user.

[0019] To improve the convenience and flexibility of user-inputted voice description segments, optionally, voice description segments can be collected when the service application is in sleep recording mode. Sleep recording mode is activated either after the user triggers the sleep recording interface configured on the service application's access page or when the service application identifies the target gesture category by performing gesture category recognition on the user's gesture image; for example... Figure 3 The voice capture page shown is in sleep recording mode. After the user triggers the voice capture control 301 on the voice capture page, the service application captures the user's voice description segments while in sleep mode; for example... Figure 4 As shown on the voice input page, users can end the input of a voice description segment by sliding the control deployed on the page. The service application then stops collecting the voice description segment. The "red wooden house and winding path", "aircraft", and "pink Ferris wheel" on the voice input page are virtual objects obtained by the service application in real time by converting the voice description segment into a voice description text and then performing virtual object recognition on the description text.

[0020] Specifically, the service application can call the image acquisition component to acquire user gesture images. The user gesture images can be one or more images. In the case of multiple images, the user gesture images can be a sequence of user gesture images. The image acquisition component can be a front-facing camera and / or a rear-facing camera.

[0021] In practical applications, users may need to record voice descriptions of their sleep state upon waking, or they may need to fall asleep quickly. To address this, and to ensure the completeness and comprehensiveness of the collected voice descriptions of the user during sleep, a target gesture category can be set. Users can activate the service application's sleep recording mode by performing the target gesture corresponding to that category. In one optional implementation of this embodiment, the following operations are performed during gesture category recognition of the user's gesture image: Hand keypoint detection is performed on user gesture images to obtain the coordinates of hand keypoints; The spatial relationship of hand key points is identified based on their coordinates, and the spatial positional relationship of the hand key points is obtained. The gesture category is then determined based on the spatial positional relationship.

[0022] Among them, the spatial positional relationship of key points of the hand can be the spatial positional relationship of key points of the fingertips.

[0023] Specifically, in the process of detecting hand key points and obtaining hand key point coordinates in a user gesture image, the user gesture image can be input into a detection engine to detect hand key points and obtain hand key point coordinates; the detection engine here can be a hand detection model, such as MediaPipe Hands (hand key point detection model); in the process of determining the gesture category based on spatial position relationship, the spatial position relationship can be used to determine whether the gesture category is the target gesture category.

[0024] To ensure user voice privacy and security, and to ensure that voice description segments during sleep are transmitted locally and processed locally, the voice description segments can be optionally deleted after the service application performs voice conversion to obtain the description text. In one optional implementation of this embodiment, voice conversion is performed in the following way: Input the speech description segment into the speech recognition engine to obtain the initial description text; The initial description text is subjected to fuzzy recognition and semantic completion to obtain the description text.

[0025] The speech recognition engine can be a lightweight ASR (Automatic Speech Recognition) engine.

[0026] It should be noted that the operation of obtaining the description text of the user's voice description segment in sleep state uploaded by the service application can be replaced by obtaining the description text of the user's voice description segment in sleep state, and forming a new implementation method with other processing steps provided in this embodiment.

[0027] Step S204: Perform virtual object recognition on the description text to obtain virtual objects, and construct a visualization scene based on the virtual objects to obtain visualization scene data.

[0028] The above-mentioned service application uploads a user's voice description fragment in a sleep state. In this step, in order to realize the user's emotional intent through the voice description fragment and realize subsequent group interaction, virtual object recognition is performed on the description text to obtain virtual objects. Based on the virtual objects, a visualization scene is constructed to obtain visualization scene data.

[0029] The virtual object described in this embodiment can be a virtual scene object in the descriptive text, specifically a virtual scene object describing the user's brain activity in a sleep state. For example, if the descriptive text is "I dreamed that my dad and I were exploring a glowing forest. We found a strange tree that could sing, and it was covered with twinkling stars. I felt incredibly happy and full of energy," the virtual objects obtained by performing virtual object recognition on the descriptive text can include "with dad," "glowing forest," and "strange tree that could sing."

[0030] Visualized scene data can be visualized virtual scene data that reconstructs the brain activity scene of a user in a sleep state based on virtual objects. Visualized scene data can be virtual scene videos and / or virtual scene images.

[0031] In practice, virtual objects are identified from the descriptive text to obtain virtual objects, and visualization scene data is constructed based on the virtual objects. The visualization scene data can then be sent to the service application. The service application can render and display the visualization scene data sent by the application server, and can also upload the user's interaction commands for the visualization scene data to the application server.

[0032] In the specific execution process, when obtaining virtual objects by performing virtual object recognition on the description text, the description text can be used to perform named entity recognition to obtain key entities as virtual objects, or the description text can be used to perform keyword recognition to obtain scene keywords as virtual objects.

[0033] In practical implementation, to construct a scene atmosphere that better matches the actual brain activity of users during sleep, and to make the constructed visual scene data more closely resemble the actual brain activity scene, visual themes can be added to the scene reconstruction prompts. In an optional implementation provided in this embodiment, during the process of obtaining visual scene data by constructing a visual scene based on virtual objects, visual themes are obtained by matching visual themes based on user status tags extracted from the description text, and virtual scene images are obtained by reconstructing virtual scenes based on visual themes and virtual scene objects. Specifically, the following operations can be performed: Visual themes are obtained by performing visual theme matching based on user status tags extracted from descriptive text, and scene reconstruction prompts are obtained by constructing prompts based on visual themes and virtual scene objects; The scene reconstruction prompts are input into the generative model to reconstruct the virtual scene and obtain the virtual scene image.

[0034] Among them, user status tags can be user emotion tags, such as user status tags being fear or pleasure; visual themes can be visual style keywords, such as visual style keywords being surreal style; generative models can be large models of text-to-image or generative models and / or large models of text-to-video or generative models; generative models can also be text-to-image models or text-to-video models.

[0035] Specifically, in the process of obtaining visual topics by visual topic matching based on user status tags extracted from descriptive text, the mapped visual topics can be queried in the visual topic library based on the user status tags extracted from descriptive text.

[0036] For example Figure 5 The scene generation page shown displays virtual object images of "cabin" and "aircraft". Virtual scene images or videos can be generated based on the virtual object images. Users can select the time to generate the brain activity scene through the time selection area at the bottom of the page.

[0037] To improve the effectiveness of virtual scene reconstruction and make the reconstructed virtual scene images more realistic, with more natural image textures, and to better match the brain activity of a user during sleep, this embodiment provides an optional implementation method for virtual scene reconstruction as follows: Semantic vectors are obtained by semantically encoding virtual scene objects and visual themes in scene reconstruction prompts, and then the semantic vectors are input into a diffusion network to denoise random noise to obtain visual scene features. The visual scene features are decoded to obtain virtual scene images.

[0038] The random noise can be random Gaussian noise; the diffusion network can be a diffusion module, such as Dream Diffusion or Stable Diffusion.

[0039] Specifically, in the process of inputting semantic vectors into a diffusion network to denoise random noise and obtain visual scene features, noise prediction and noise removal can be performed on random noise based on semantic vectors to obtain visual scene features; the above semantic encoding can be performed by an encoder, and the decoding process can be performed by a decoder.

[0040] After obtaining the visualization scene data, in order for users to perceive the visualization scene data in a timely manner and to meet users' editing needs for the visualization scene data, making the visualization scene data more closely resemble the user's real brain activity scenario, in an optional implementation of this embodiment, the visualization scene data can be distributed to the service application. Accordingly, after the service application renders and displays the visualization scene data, it can also perform the following operations: The system identifies the attributes of user image editing commands for virtual scene images to obtain the editing command attributes. If the editing instruction attribute is a content editing attribute, obtain the coordinates of the selected area of ​​the virtual scene image by the user and the content editing information, and upload them to the application server.

[0041] Among them, the editing instruction attribute can be an instruction attribute for editing the virtual scene image, such as the editing instruction attribute including content editing attribute and / or style editing attribute; the selection area coordinates can be the coordinates of the selected area in the virtual scene image by the user; the content editing information can be content editing requirement information or content change requirement information, specifically the editing requirement information or change requirement information that the user wants to edit or change the selected area of ​​the virtual scene image.

[0042] To update the virtual scene image in a timely manner according to user needs, in one optional implementation of this embodiment, after receiving the user's selected area coordinates and content editing information for the virtual scene image, the following operations are performed: The service application uploads the coordinates of the user's selected area and content editing information for the virtual scene image; Based on the selected area coordinates and content editing information, the virtual scene image is edited to obtain an edited scene image, which is then sent to the service application to update the virtual scene image.

[0043] It should be noted that after obtaining the visualization scene data by constructing a visualization scene based on virtual objects, the system can also detect the user's selected area coordinates and content editing information uploaded by the service application. Based on the selected area coordinates and content editing information, the system can edit the virtual scene image to obtain an edited scene image and send it to the service application to update the virtual scene image. In this embodiment, the visualization scene data can be a virtual scene image and / or a virtual scene video, and the virtual scene image can be replaced with a virtual scene video.

[0044] In practical applications, users may only have editing needs for the style of virtual scene images. To address this, and to improve the response speed to user style editing requests, allowing users to promptly perceive the updated style of the virtual scene image, in one optional implementation of this embodiment, after recognizing the instruction attribute, the service application can, if the instruction attribute is a style editing attribute, perform image style migration on the virtual scene image based on the target image style selected by the user in the image style set, obtaining a migrated scene image and updating the virtual scene image. Specifically, after recognizing the instruction attribute, the service application can also perform the following operations: If the editing instruction attribute is a style editing attribute, determine the target image style selected by the user from the image style set; The virtual scene image and the target image style are input into a lightweight style transfer model to perform image style transfer, obtain the transferred scene image, and update the virtual scene image.

[0045] Image style transfer can be image style transformation.

[0046] Step S206: Obtain user emotional intent by parsing the virtual object, and construct a group interaction task for the user group based on the user emotional intent.

[0047] The above-mentioned virtual object recognition of the descriptive text is used to obtain virtual objects, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects. In this step, the emotional intent of the user is obtained by analyzing the emotional intent of the virtual objects, and group interaction tasks are constructed based on the user's emotional intent. In this way, the subconscious emotional intent of the user is mined from the user's voice description fragments in the sleep state, and further group interaction is carried out, thereby enhancing group relationships and deepening group connections through group interaction.

[0048] The user's emotional intent described in this embodiment can be user demand information or user emotional demand information. Continuing with the previous example, the virtual objects include "with Dad", "glowing forest", and "singing strange tree". The user's emotional intent is "I hope to create a 'glowing tree' of our own in reality with Dad to commemorate this wonderful dream adventure". The group interaction task can be a family group interaction task or a friend group interaction task. For example, the group interaction task is a weekend family tree planting task.

[0049] In practice, during the process of obtaining user emotional intent by parsing the emotional intent of virtual objects, the virtual objects can be input into a large language model for emotional intent parsing to obtain user emotional intent; for example, the large language model here is an LLM (Large Language Model), or the virtual objects can be input into an emotional parsing model for emotional intent parsing to obtain user emotional intent; the emotional parsing model here can be obtained by fine-tuning the large language model using a psychology knowledge base.

[0050] In practical applications, the virtual objects identified from the descriptive text may be diverse. These virtual objects may reflect other users with whom the user wishes to engage in collaborative interaction. Therefore, to delve deeper into identifying other users with whom the user wishes to interact, i.e., identifying group members with whom the user wishes to perform collaborative tasks, this embodiment provides an optional implementation where, during the process of obtaining the user's emotional intent through emotional intent parsing based on virtual objects, user psychological data is obtained through user psychological analysis based on each virtual scene object. The user's emotional intent is then generated based on this user psychological data and the group members identified from the target virtual scene object. Specifically, the following operations can be performed: User psychological data is obtained by performing user psychological analysis on each virtual scene object, and group members are obtained by identifying group members based on the target virtual scene object; User emotional intent is generated based on user psychological data and group members.

[0051] Specifically, in the process of obtaining user psychological data through user psychological analysis based on each virtual scene object, the emotional attribute tags mapped to each virtual scene object can be queried, and user psychological state data can be generated based on the emotional attribute tags, daytime events, historical psychological state data, and / or user state tags extracted from the descriptive text; where daytime events can be events that occur during the day and are associated with the virtual scene object; emotional attribute tags can be tags that represent emotional attributes, such as confusion as the mapped emotional attribute tag for a maze virtual scene object; after obtaining user psychological data through user psychological analysis based on each virtual scene object, a psychological analysis report of the user's brain activity scene can be constructed based on the user psychological data, or an emotional intent report can be constructed based on the user's emotional intent; for example, Figure 6 The emotional intention report page shows the scene pleasure index of the user's brain activity scenario. Clicking on "Inner Child", "Loving Subconscious" and "Warning" will display different user psychological data respectively.

[0052] To help users realize their emotional intentions and enhance group relationships, this embodiment provides an optional implementation method in which the following operations are performed during the process of constructing scene interaction tasks for user groups based on user emotional intentions to obtain group interaction tasks: Based on user emotional intent, current time attributes, and visual scene data, initial task matching is performed in the task knowledge base to obtain group task topics; Based on user geographic location and user attribute data, group tasks are generated under the group task theme of the user group composed of group members to obtain group interaction tasks.

[0053] Among them, the group task theme can be a theme that represents the task direction of the group task, such as the group task theme being family tree planting; the current time attribute can be the current season data, specifically data that represents which season we are currently in; user attribute data can be the date of birth and / or the learning stage.

[0054] Specifically, in the process of generating group tasks under the group task theme of a user group composed of group members based on user geographic location and user attribute data, the task content of the group task items configured under the group task theme can be determined based on user geographic location and user attribute data, and the group interaction task can be constructed based on the task content of each group task item.

[0055] Step S208: If an interaction instruction for the visualized scene data is detected, perform task interaction processing for the group interaction task based on the visualized scene data.

[0056] The above-mentioned process involves parsing the user's emotional intent based on the virtual object, and then constructing a group interaction task based on the user's emotional intent. In this step, if an interaction instruction from the visualized scene data is detected, the task interaction processing of the group interaction task is performed based on the visualized scene data.

[0057] In a specific implementation, in one optional embodiment provided by this example, during the task interaction processing of group interaction tasks based on visualized scene data, the following operations are performed: Based on visualized scene data, group interaction tasks, and emotional intent reports, task invitation reminders are generated and pushed to group members of the user's group; Collaborative execution of group interactive tasks is achieved based on group members' confirmation commands for task invitation reminders.

[0058] Specifically, task invitation reminders can be generated and pushed to group members based on visualized scene data, group interaction tasks, and emotional intent reports. Members' terminals can be redirected to the service application based on the task invitation reminder's trigger command. Within the service application, a task invitation is constructed based on the visualized scene data, group interaction tasks, and / or emotional intent reports, and then rendered and displayed on the group interaction page. Group members can edit the task invitation on the group interaction page, and the application server can update the group interaction task based on the edited information. Group members can collaboratively execute updated group interaction tasks. A user group can include the user corresponding to the voice description segment, as well as the user's associated users, such as family and friends.

[0059] During the collaborative execution of group interaction tasks, a task resource space corresponding to the group interaction task can be created, and the space configuration of the task resource space can be based on the task execution images uploaded by group members, such as the task resource space being a task album.

[0060] For example Figure 7 On the scene sharing page shown, after a user clicks the "Share with Parents" control, the application server generates a task invitation reminder based on the visualized scene data, group interaction tasks, and emotional intent reports, and pushes it to group members. The scene sharing page can also display a collection of virtual scenes belonging to the user, which is constructed based on the user's historical virtual scene images; for example... Figure 8 On the group interaction page shown, after receiving a task invitation reminder, parents can view visualized scene data, or click "Fulfill Dreams Together" to collaboratively execute group interaction tasks, including this week's hike, rescue together, and winking at you.

[0061] It should be noted that the user data obtained in this manual, such as the user's voice descriptions during sleep, has been authorized by the user and does not involve user privacy. Specifically, authorization can be granted during the user's first access to the application, the interactive subroutines within the application, or the interactive application itself, or each time the user accesses the application, the interactive subroutines within the application, or the interactive application itself.

[0062] It should be added that each optional implementation method and each feasible execution method in steps S202 to S208 provided in this embodiment can be executed independently as needed, or they can be combined and referenced with each other. At the same time, each specific execution step in each optional implementation method or each feasible execution method can also be executed independently or combined as needed. The execution conditions of "if" or "under what circumstances" involved in each step or operation can be directly deleted, and subsequent operations can be executed. This embodiment does not make specific limitations on this.

[0063] It should also be added that, depending on the actual application scenario, step S202 and any of the subsequent steps S204 to S208 can be deleted, or any feature in any step can be deleted. For example, "uploaded by service application" in step S202 can be deleted, and the execution order of steps S202 to S208 can also be arbitrary.

[0064] The implementation process of the data processing method provided above can be executed by the application server. The implementation process of the other data processing method provided in the following method embodiment can be executed by the service application. The two can cooperate with each other during the execution process. Therefore, when reading the above implementation process, you can refer to the corresponding content of the following other data processing method embodiment. Similarly, when reading the following other data processing method embodiment, you can also refer to the corresponding content of the above method embodiment.

[0065] The following description uses the application of a data processing method provided in this embodiment in a home interaction scenario as an example to further illustrate the data processing method provided in this embodiment. (See also...) Figure 9 The data processing method applied to home interaction scenarios can be applied to the application server and includes the following steps.

[0066] Step S904: Obtain the description text of the scene description speech of a specific user's brain activity in a sleep state uploaded by the service application.

[0067] Step S906: Perform virtual object recognition on the description text to obtain virtual scene objects, perform visual theme matching based on the user status tags extracted from the description text to obtain visual themes, and construct prompt words based on visual themes and virtual scene objects to obtain scene reconstruction prompt words.

[0068] Step S908: Input the scene reconstruction prompts into the generative model to reconstruct the virtual scene and obtain a virtual scene image.

[0069] Step S910: Obtain the user's emotional intent by parsing the virtual scene object, and construct the scene interaction task for the family group based on the user's emotional intent to obtain the family group task.

[0070] Step S912: Generate a group sharing message based on the family group task and the virtual scene image and send it to the service application.

[0071] Step S918: Based on the selected area coordinates and content editing information uploaded by the service application, the virtual scene image is edited to obtain an edited scene image, which is then sent to the service application to update the virtual scene image.

[0072] Step S922: After detecting the interactive instruction to edit the scene image submitted by the service application, generate a task invitation reminder based on the edit scene image, the family group task, and the emotional intent report, and push it to the family group members.

[0073] Step S924: Collaborative execution of family group tasks based on the confirmation instructions from family group members for the task invitation reminder.

[0074] Steps S904 to S912, S918, and S922 to S924 provided in this embodiment are executed by the application server. It should be noted that the steps S904 to S912, S918, and S922 to S924 executed by the application server can cooperate with steps S902, S914 to S916, and S920 executed by the service application in the following embodiment during execution. Therefore, when reading this embodiment, please refer to the corresponding content of steps S902, S914 to S916, and S920 provided in the following method embodiment, and when reading the following method embodiment, please refer to the corresponding content of steps S904 to S912, S918, and S922 to S924 provided in this embodiment.

[0075] It should be noted that any one or more of steps S904 to S912, S918, and S922 to S924 can be replaced with the corresponding technical means provided by steps S202 to S208 as needed for implementation and deployment. Furthermore, any one or more of steps S904 to S912, S918, and S922 to S924 can be combined to form a new implementation method as needed for implementation and deployment. In addition, any one or more of steps S904 to S912, S918, and S922 to S924 can also be combined with one or more of the steps provided by steps S202 to S208 to form a new implementation method, or combined with one or more of the optional implementation methods provided by steps S202 to S208 to form a new implementation method, as needed for actual deployment. These will not be elaborated on here.

[0076] One or more embodiments of another data processing method provided in this specification are as follows: Reference Figure 10 The data processing method provided in this embodiment can be applied to service applications, specifically including steps S1002 to S1004.

[0077] Step S1002: Convert the user's voice description segment in the sleep state into speech and upload the resulting description text to the application server.

[0078] The service application described in this embodiment can be an application or a subroutine, specifically a payment application or a subroutine within a payment application; the service application can be in a specific user service state or a specific user mode, such as a service application in a youth mode.

[0079] The voice description segment may include sleep talking segments of the user while asleep or a scene description of the user's brain activity while asleep; here, the scene of the user's brain activity while asleep can be a dream scene, and the voice description segment can be input by the user while asleep or while awake. That is, the user can input sleep talking segments while asleep, and the user can input scene descriptions of the user's brain activity while asleep while awake.

[0080] In practice, the user's voice description segment during sleep can be transcribed into speech, and the resulting text description can be uploaded to the application server. The application server can then retrieve the text description of the user's sleep voice description segment uploaded by the service application. Furthermore, it can perform virtual object recognition on the text description to obtain virtual objects, and construct a visual scene based on these virtual objects to obtain visual scene data. This visual scene data can then be distributed to the service application. The user can be a specific user, such as a teenager, a child, an elderly person, or a pregnant woman.

[0081] To improve the convenience and flexibility of user-inputted voice description segments, optionally, the voice description segments can be collected when the service application is in sleep recording mode. Sleep recording mode is activated after the user triggers the sleep recording interface configured on the service application's access page or when the service application identifies the target gesture category by performing gesture category recognition on the user's gesture image. Specifically, the service application can call an image acquisition component to acquire user gesture images. The user gesture images can be one or more images. In the case of multiple images, the user gesture images can be a sequence of user gesture images. The image acquisition component can be a front-facing camera and / or a rear-facing camera.

[0082] In practical applications, users may need to record a voice description of their sleep state upon waking, or they may need to fall asleep quickly. To address this, and to ensure the completeness and comprehensiveness of the collected voice descriptions during sleep, a target gesture category can be set. Users can activate the sleep recording mode by performing the target gesture corresponding to that category. During the gesture category recognition process, the service application can perform the following operations: Hand keypoint detection is performed on user gesture images to obtain the coordinates of hand keypoints; The spatial relationship of hand key points is identified based on their coordinates, and the spatial positional relationship of the hand key points is obtained. The gesture category is then determined based on the spatial positional relationship.

[0083] Among them, the spatial positional relationship of key points of the hand can be the spatial positional relationship of key points of the fingertips.

[0084] Specifically, in the process of detecting hand key points and obtaining hand key point coordinates in a user gesture image, the user gesture image can be input into a detection engine to detect hand key points and obtain hand key point coordinates; the detection engine here can be a hand detection model, such as MediaPipe Hands (hand key point detection model); in the process of determining the gesture category based on spatial position relationship, the spatial position relationship can be used to determine whether the gesture category is the target gesture category.

[0085] To ensure user voice privacy and security, and to ensure that voice description segments during sleep are transmitted locally and processed locally, the voice description segments can be optionally deleted after the service application converts the voice description segments into text. Specifically, the service application can perform the following operations during the voice conversion process: Input the speech description segment into the speech recognition engine to obtain the initial description text; The initial description text is subjected to fuzzy recognition and semantic completion to obtain the description text.

[0086] The speech recognition engine can be a lightweight ASR (Automatic Speech Recognition) engine.

[0087] Step S1004: Render and display the visualized scene data issued by the application server, and upload the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing of group interaction tasks based on the visualized scene data.

[0088] Optionally, the group interaction task is obtained by constructing a scenario interaction task for a user group based on the user's emotional intent, and the user's emotional intent is obtained by identifying the emotional intent of virtual objects obtained from the description text.

[0089] In practice, during the task interaction processing of group interaction tasks based on visualized scene data, the application server can perform the following operations: Based on visualized scene data, group interaction tasks, and emotional intent reports, task invitation reminders are generated and pushed to group members; Collaborative execution of group interactive tasks is achieved based on group members' confirmation commands for task invitation reminders.

[0090] Specifically, the application server can generate task invitation reminders based on visualized scene data, group interaction tasks, and emotional intent reports, and push them to group members. Members' terminals can be redirected to the service application based on the task invitation reminder's trigger command. Within the service application, a task invitation is constructed based on the visualized scene data, group interaction tasks, and / or emotional intent reports, and then rendered and displayed on the group interaction page. Group members can edit the task invitation on the group interaction page, and the application server can update the group interaction task based on the edited information. Group members can collaboratively execute updated group interaction tasks. The user group can include the user corresponding to the voice description segment, as well as the user's associated users, such as family and friends.

[0091] During the collaborative execution of group interaction tasks, the application server can create a task resource space corresponding to the group interaction task, and configure the space of the task resource space based on the task execution images uploaded by group members, such as the task resource space being a task album.

[0092] The following description uses the application of a data processing method provided in this embodiment in a home interaction scenario as an example to further illustrate the data processing method provided in this embodiment. (See also...) Figure 9 The data processing method applied to home interaction scenarios can be applied to service applications and includes the following steps.

[0093] Step S902: The speech describing the brain activity scenario of a specific user in a sleep state is converted into speech, and the resulting descriptive text is uploaded to the application server.

[0094] Step S914: Perform instruction attribute recognition on the image editing instructions of a specific user for the virtual scene image in the group sharing message issued by the application server, and obtain the editing instruction attributes.

[0095] Step S916: If the editing instruction attribute is a content editing attribute, obtain the coordinates of the selected area of ​​the virtual scene image by the specific user and the content editing information and upload them to the application server.

[0096] Step S920: Update the virtual scene image according to the edit scene image issued by the application server, and submit the specific user's interaction command for editing the scene image to the application server.

[0097] It should be noted that any one or more of steps S902, S914 to S916, and S920 can be replaced by the corresponding technical means provided in steps S1002 to S1004 as needed for implementation and deployment. Furthermore, any one or more of steps S902, S914 to S916, and S920 can be combined to form a new implementation method as needed for implementation and deployment. Additionally, any one or more of steps S902, S914 to S916, and S920 can also be combined with one or more of the steps provided in steps S1002 to S1004 to form a new implementation method, or combined with one or more optional implementation methods provided in steps S1002 to S1004 to form a new implementation method, as needed for actual deployment. These details will not be elaborated upon here.

[0098] The following is an embodiment of a data processing device provided in this specification: In the above embodiments, a data processing method is provided, and correspondingly, a data processing device is also provided, which will be described below with reference to the accompanying drawings.

[0099] Reference Figure 11 This illustration shows a schematic diagram of an embodiment of a data processing device provided in this embodiment.

[0100] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0101] This embodiment provides a data processing apparatus, including: The text acquisition module 1102 is configured to acquire the descriptive text of the user's voice description segment in sleep state uploaded by the service application; The object recognition module 1104 is configured to perform virtual object recognition on the description text to obtain virtual objects, and to construct a visualization scene based on the virtual objects to obtain visualization scene data; The intent parsing module 1106 is configured to perform emotional intent parsing based on the virtual object to obtain the user's emotional intent, and to construct a scene interaction task for the user group based on the user's emotional intent to obtain a group interaction task. The task interaction module 1108 is configured to perform task interaction processing of the group interaction task based on the visualized scene data if an interaction instruction is detected in the visualized scene data.

[0102] Another embodiment of the data processing device provided in this specification is as follows: In the above embodiments, another data processing method is provided, and correspondingly, another data processing device is also provided, which will be described below with reference to the accompanying drawings.

[0103] Reference Figure 12 This illustration shows a schematic diagram of an embodiment of a data processing device provided in this embodiment.

[0104] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0105] This embodiment provides a data processing apparatus, including: The speech conversion module 1202 is configured to convert the user's speech description segment in the sleep state into speech and upload the description text obtained by speech conversion to the application server. The instruction upload module 1204 is configured to render and display the visual scene data issued by the application server, and upload the user's interaction instructions on the visual scene data to the application server, so as to perform task interaction processing of group interaction tasks based on the visual scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

[0106] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0107] The following is an embodiment of a data processing device provided in this specification: Corresponding to the data processing method described above, based on the same technical concept, one or more embodiments of this specification also provide a data processing apparatus for performing the data processing method provided above. Figure 13 This is a schematic diagram of the structure of a data processing device provided for one or more embodiments of this specification.

[0108] This embodiment provides a data processing device, including: like Figure 13As shown, device 1300 mainly consists of a communication interface 1302, a user interface 1304, a processor 1306, and a data storage 1308. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1310. The communication interface 1302 enables device 1300 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1302 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1302 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1302 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1302 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces. The user interface 1304 includes receiving user input and providing output to the user. Therefore, user interface 1304 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1304 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1304 may include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, device 1300 may support remote access from other devices via communication interface 1302 or another physical interface (not shown). User interface 1304 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1304 may also be configured as a display device for rendering or displaying text fragments.

[0109] Processor 1306 may include one or more general-purpose processors and / or special-purpose processors. Data storage 1308 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1306. Data storage 1308 may include removable and non-removable components.

[0110] Processor 1306 is capable of executing program instructions 1318 (e.g., compiled or uncompiled program logic and / or machine code) stored in data store 1308 to perform the various functions described herein. Data store 1308 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1300, enable device 1300 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1318 by processor 1306 may result in processor 1306 using data 1312. For example, program instructions 1318 may include an operating system 1322 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1300 and one or more application programs 1320 (e.g., a browser, social application, or game application). Similarly, data 1312 may include operating system data 1316 and application data 1314. Operating system data 1316 is primarily accessible to operating system 1322, while application data 1314 is primarily accessible to one or more application programs 1320. Application data 1314 may reside in a file system visible or hidden to the user of device 1300. Application 1320 may communicate with operating system 1322 via one or more application programming interfaces (APIs). These APIs facilitate application 1320 reading and / or writing application data 1314, transmitting or receiving information via communication interface 1302, receiving or displaying information on user interface 1304, etc. In some terms, application 1320 may be simply referred to as an "app". Furthermore, application 1320 may be downloaded to device 1300 through one or more online app stores or app markets. However, applications may also be installed on device 1300 in other ways, such as through a web browser or a physical interface on device 1300 (e.g., a USB port).

[0111] In one specific embodiment, the data processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the data processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Get the descriptive text of the user's voice description segment in sleep state uploaded by the service application; Virtual objects are obtained by performing virtual object recognition on the description text, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects; Based on the virtual object, the user's emotional intent is obtained by parsing the emotional intent, and based on the user's emotional intent, a scene interaction task is constructed for the user group to obtain a group interaction task. If an interaction instruction is detected in the visualized scene data, the task interaction processing of the group interaction task is performed based on the visualized scene data.

[0112] Another embodiment of the data processing device provided in this specification is as follows: Corresponding to the other data processing method described above, based on the same technical concept, one or more embodiments of this specification also provide another data processing apparatus for performing the other data processing method provided above. Figure 14 This is a schematic diagram of the structure of a data processing device provided for one or more embodiments of this specification.

[0113] This embodiment provides a data processing device, including: like Figure 14As shown, device 1400 mainly consists of a communication interface 1402, a user interface 1404, a processor 1406, and a data storage 1408. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1410. The communication interface 1402 enables device 1400 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1402 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1402 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1402 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1402 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces. The user interface 1404 includes receiving user input and providing output to the user. Therefore, user interface 1404 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1404 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1404 may include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, device 1400 may support remote access from other devices via communication interface 1402 or another physical interface (not shown). User interface 1404 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1404 may also be configured as a display device for rendering or displaying text fragments.

[0114] Processor 1406 may include one or more general-purpose processors and / or special-purpose processors. Data storage 1408 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 1406. Data storage 1408 may include removable and non-removable components.

[0115] Processor 1406 is capable of executing program instructions 1418 (e.g., compiled or uncompiled program logic and / or machine code) stored in data store 1408 to perform the various functions described herein. Data store 1408 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1400, enable device 1400 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1418 by processor 1406 may result in processor 1406 using data 1412. For example, program instructions 1418 may include an operating system 1422 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1400 and one or more application programs 1420 (e.g., a browser, social application, or game application). Similarly, data 1412 may include operating system data 1416 and application data 1414. Operating system data 1416 is primarily accessible to operating system 1422, while application data 1414 is primarily accessible to one or more application programs 1420. Application data 1414 may reside in a file system visible or hidden from the user of device 1400. Application 1420 may communicate with operating system 1422 via one or more application programming interfaces (APIs). These APIs facilitate application 1420 reading and / or writing application data 1414, transmitting or receiving information via communication interface 1402, receiving or displaying information on user interface 1404, etc. In some terms, application 1420 may be simply referred to as an "app". Furthermore, application 1420 may be downloaded to device 1400 through one or more online app stores or app markets. However, applications may also be installed on device 1400 in other ways, such as through a web browser or a physical interface on device 1400 (e.g., a USB port).

[0116] In one specific embodiment, the data processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the data processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: The system converts the user's voice descriptions during sleep into speech and uploads the resulting text to the application server. The application server renders and displays the visualized scene data, and uploads the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

[0117] This specification provides an embodiment of a computer-readable storage medium as follows: Corresponding to the data processing method described above, and based on the same technical concept, one or more embodiments of this specification also provide a computer-readable storage medium.

[0118] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed, perform the following steps: Get the descriptive text of the user's voice description segment in sleep state uploaded by the service application; Virtual objects are obtained by performing virtual object recognition on the description text, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects; Based on the virtual object, the user's emotional intent is obtained by parsing the emotional intent, and based on the user's emotional intent, a scene interaction task is constructed for the user group to obtain a group interaction task. If an interaction instruction is detected in the visualized scene data, the task interaction processing of the group interaction task is performed based on the visualized scene data.

[0119] It should be noted that the embodiments of a computer-readable storage medium described in this specification and the embodiments of a data processing method described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0120] Another embodiment of a computer-readable storage medium provided in this specification is as follows: In response to the other data processing method described above, and based on the same technical concept, one or more embodiments of this specification also provide another computer-readable storage medium.

[0121] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed, perform the following steps: The system converts the user's voice descriptions during sleep into speech and uploads the resulting text to the application server. The application server renders and displays the visualized scene data, and uploads the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

[0122] It should be noted that the embodiments of another computer-readable storage medium described in this specification and the embodiments of another data processing method described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0123] This specification provides an example of a computer program product as follows: Corresponding to the data processing method described above, and based on the same technical concept, one or more embodiments of this specification also provide a computer program product.

[0124] A computer program product includes a computer program / instructions that, when executed by a processor, perform the following steps: Get the descriptive text of the user's voice description segment in sleep state uploaded by the service application; Virtual objects are obtained by performing virtual object recognition on the description text, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects; Based on the virtual object, the user's emotional intent is obtained by parsing the emotional intent, and based on the user's emotional intent, a scene interaction task is constructed for the user group to obtain a group interaction task. If an interaction instruction is detected in the visualized scene data, the task interaction processing of the group interaction task is performed based on the visualized scene data.

[0125] It should be noted that the embodiments of a computer program product described in this specification and the embodiments of a data processing method described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0126] Another example of a computer program product provided in this specification is as follows: Corresponding to the other data processing method described above, and based on the same technical concept, one or more embodiments of this specification also provide another computer program product.

[0127] A computer program product includes a computer program / instructions that, when executed by a processor, perform the following steps: The system converts the user's voice descriptions during sleep into speech and uploads the resulting text to the application server. The application server renders and displays the visualized scene data, and uploads the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

[0128] It should be noted that the embodiments of another computer program product described in this specification and the embodiments of another data processing method described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0129] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments. For example, the device embodiment, equipment embodiment and computer-readable storage medium embodiment are all similar to the method embodiment, so the description is relatively simple. When reading the relevant content of the device embodiment, equipment embodiment and computer-readable storage medium embodiment, please refer to the description of the method embodiment.

[0130] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

[0131] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0132] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multiple data processing and parallel processing are also possible or may be advantageous.

[0133] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0134] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0135] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0136] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0137] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable test processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable test processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable test processing equipment to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable test processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0142] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0143] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0144] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of features includes not only those features but also other features not expressly listed, or features inherent to such process, method, article, or apparatus. Without further limitations, a feature defined by the phrase "comprising one..." does not exclude the presence of other identical features in the process, method, article, or apparatus that includes said feature.

[0145] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0146] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0147] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A data processing method, comprising: Get the descriptive text of the user's voice description segment in sleep state uploaded by the service application; Virtual objects are obtained by performing virtual object recognition on the description text, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects; Based on the virtual object, the user's emotional intent is obtained by parsing the emotional intent, and based on the user's emotional intent, a scene interaction task is constructed for the user group to obtain a group interaction task. If an interaction instruction is detected in the visualized scene data, the task interaction processing of the group interaction task is performed based on the visualized scene data.

2. The data processing method according to claim 1, wherein the step of constructing a visualization scene based on the virtual object to obtain visualization scene data includes: Visual themes are obtained by performing visual theme matching based on user status tags extracted from the description text, and scene reconstruction prompts are obtained by constructing prompts based on the visual themes and virtual scene objects. The scene reconstruction prompts are input into a generative model to reconstruct the virtual scene and obtain a virtual scene image.

3. The data processing method according to claim 2, wherein the virtual scene reconstruction is performed in the following manner: The virtual scene objects and visual themes in the scene reconstruction prompts are semantically encoded to obtain semantic vectors, and the semantic vectors are input into a diffusion network to denoise random noise to obtain visual scene features; The virtual scene image is obtained by decoding the visual scene features.

4. The data processing method according to claim 1, wherein obtaining the user's emotional intent by parsing the virtual object includes: User psychological data is obtained by performing user psychological analysis on each virtual scene object, and group members are obtained by identifying group members based on the target virtual scene object; The user's emotional intent is generated based on the user's psychological data and the group members.

5. The data processing method according to claim 4, wherein the step of constructing a group interaction task based on the user's emotional intent for the user group includes: Based on the user's emotional intent, current time attribute, and the visualized scene data, an initial task matching is performed in the task knowledge base to obtain the group task theme; Based on user geographic location and user attribute data, group tasks are generated under the group task theme of the user group composed of the group members to obtain the group interaction task.

6. According to the data processing method of claim 1, after the service application renders and displays the visualization scene data, it further performs the following operations: The image editing commands given by the user for the virtual scene image are identified by command attribute recognition to obtain the editing command attributes; If the editing instruction attribute is a content editing attribute, obtain the coordinates of the selected area of ​​the virtual scene image by the user and the content editing information and upload them to the application server.

7. The data processing method according to claim 6, further comprising: Receive the coordinates of the selected area and content editing information of the virtual scene image uploaded by the service application; Based on the selected area coordinates and the content editing information, the virtual scene image is edited to obtain an edited scene image, which is then sent to the service application to update the virtual scene image.

8. According to the data processing method of claim 6, after the service application performs the instruction attribute identification, it further performs the following operation: If the editing instruction attribute is a style editing attribute, determine the target image style selected by the user in the image style set; The virtual scene image and the target image style are input into a lightweight style transfer model for image style transfer to obtain a transferred scene image and update the virtual scene image.

9. The data processing method according to claim 1, wherein the task interaction processing of the group interaction task based on the visualized scene data includes: Based on the visualized scene data, the group interaction tasks, and the emotional intent report, a task invitation reminder is generated and pushed to the group members of the user group; The group interaction task is executed collaboratively based on the group members' confirmation of the task invitation reminder.

10. The data processing method according to claim 1, wherein the service application is in a specific user service state, and the voice description segment is collected when the service application is in sleep recording mode; The sleep recording mode is activated after the user triggers the sleep recording interface configured on the access page of the service application, or when the service application obtains the target gesture category by performing gesture category recognition on the user's gesture image.

11. The data processing method according to claim 10, wherein the gesture category recognition of the user gesture image includes: Hand key point detection is performed on the user gesture image to obtain the coordinates of the hand key points; Based on the coordinates of the hand key points, spatial relationship recognition of the hand key points is performed to obtain the spatial positional relationship of the hand key points, and the gesture category is determined based on the spatial positional relationship.

12. The data processing method according to claim 1, wherein the voice description segment includes a sleep talking segment of the user in the sleep state or a scene description voice of the user's brain activity scene in the sleep state; The voice description segment is deleted after the service application performs voice conversion on the voice description segment to obtain the description text.

13. The data processing method according to claim 12, wherein the speech conversion is performed in the following manner: The speech description segment is input into a speech recognition engine for speech recognition to obtain the initial description text; The initial description text is subjected to fuzzy recognition and semantic completion to obtain the description text.

14. A data processing method, comprising: The system converts the user's voice descriptions during sleep into speech and uploads the resulting text to the application server. The application server renders and displays the visualized scene data, and uploads the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

15. A data processing apparatus, comprising: The text acquisition module is configured to acquire the descriptive text of the user's voice description segments in a sleep state uploaded by the service application; The object recognition module is configured to perform virtual object recognition on the description text to obtain virtual objects, and to construct a visualization scene based on the virtual objects to obtain visualization scene data; The intent parsing module is configured to perform emotional intent parsing based on the virtual object to obtain the user's emotional intent, and to construct a scene interaction task for the user group based on the user's emotional intent to obtain a group interaction task. The task interaction module is configured to perform task interaction processing of the group interaction task based on the visualized scene data if an interaction command is detected in the visualized scene data.

16. A data processing apparatus, comprising: The voice conversion module is configured to convert the user's voice description segments while in sleep mode into voice and upload the resulting text description to the application server. The instruction upload module is configured to render and display the visual scene data issued by the application server, and upload the user's interaction instructions on the visual scene data to the application server, so as to perform task interaction processing of group interaction tasks based on the visual scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

17. A data processing apparatus, comprising: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to: Get the descriptive text of the user's voice description segment in sleep state uploaded by the service application; Virtual objects are obtained by performing virtual object recognition on the description text, and visualization scene data is obtained by constructing a visualization scene based on the virtual objects; Based on the virtual object, the user's emotional intent is obtained by parsing the emotional intent, and based on the user's emotional intent, a scene interaction task is constructed for the user group to obtain a group interaction task. If an interaction instruction is detected in the visualized scene data, the task interaction processing of the group interaction task is performed based on the visualized scene data.

18. A data processing apparatus, comprising: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to: The system converts the user's voice descriptions during sleep into speech and uploads the resulting text to the application server. The application server renders and displays the visualized scene data, and uploads the user's interaction instructions on the visualized scene data to the application server to perform task interaction processing for group interaction tasks based on the visualized scene data. The group interaction task is constructed based on the user's emotional intent to construct a scene interaction task for the user group. The user's emotional intent is obtained by identifying the emotional intent based on the virtual object identified from the description text.

19. A computer-readable storage medium for storing computer-executable instructions that, when executed, implement the steps of the method of claim 1 or 14.

Images