AR-based information synthesis method and system

Abstract

The present invention provides an AR-based information synthesis method and system, which relate to the field of data processing technology. In the present invention, a scene feedback data cluster to be analyzed is extracted; an ordered set of user feedback states is extracted; a reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states is determined; based on the scene feedback data cluster to be analyzed, image application users associated with the feedback action type and the reference feedback action type are analyzed, and, based on the image application user, candidate image application users are determined; among the candidate image application users, candidate image application users with identifying user representation information are determined and marked as target image application users; a scene image corresponding to the scene to be analyzed and a user image corresponding to the target image application user are merged to form a user scene synthesis image. Based on the above method, the reliability of information synthesis can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an AR-based information synthesis method and system. Background Art

[0002] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world. It widely uses multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means to simulate computer-generated text, images, three-dimensional models, music, video and other virtual information and apply them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world. For example, in some applications, it is necessary to synthesize the user image and the scene image to form a synthetic image. The user image or the scene image can be a virtual image. However, in the existing technology, there is a problem of low reliability in the synthesis of images. Summary of the invention

[0003] In view of this, an object of the present invention is to provide an information synthesis method and system based on AR to improve the reliability of information synthesis.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] An information synthesis method based on AR, comprising:

[0006] Extracting a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed, wherein the scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes a type of feedback action of an image application user on an image of the scene to be analyzed, and the image of the scene to be analyzed belongs to a scene image corresponding to the scene to be analyzed;

[0007] Extracting an ordered set of user feedback states corresponding to the scene to be analyzed, the ordered set of user feedback states comprising a plurality of user feedback states sorted according to the merits of the user feedback states, and the ordered set of user feedback states is formed by processing based on the feedback action types corresponding to the action type change path of the scene to be analyzed;

[0008] Determining a reference feedback action type corresponding to a user feedback state in the ordered set of user feedback states;

[0009] Analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type;

[0010] In the ordered set of user feedback states, a first user feedback state is determined, and, among a plurality of candidate image application users corresponding to the first user feedback state, the number of image application users corresponding to each preset user characterization information is analyzed; and, according to the number of image application users corresponding to the preset user characterization information, the identifying user characterization information corresponding to the first user feedback state is determined; and, among the candidate image application users corresponding to the second user feedback state, a candidate image application user having the identifying user characterization information is determined and marked as a target image application user, and the quality of the user feedback state corresponding to the second user feedback state is worse than that of the user feedback state corresponding to the first user feedback state;

[0011] A merging operation is performed on the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene composite image.

[0012] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the steps of analyzing the image application users associated with the feedback action type and the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application users corresponding to the user feedback state according to the feedback action type and the image application users associated with the reference feedback action type, include:

[0013] In the scene feedback data cluster to be analyzed, determining the scene feedback data to be analyzed corresponding to each image application user to form a user scene feedback data cluster corresponding to the image application user;

[0014] In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

[0015] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the user feedback states correspond to a state order relationship, and there is a positive correlation between the state order relationship and the quality of the user feedback states corresponding to the user feedback states;

[0016] The step of determining the reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states comprises: determining the current user feedback state in the ordered set of user feedback states in order based on the state sequence relationship, and determining the current reference feedback action type corresponding to the current user feedback state;

[0017] When the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, the step of marking the image application user as a candidate image application user corresponding to the user feedback state comprises:

[0018] In a case where the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type, performing a marking operation on the image application user to mark the image application user as a candidate image application user corresponding to the current user feedback state;

[0019] When the feedback action type in the user scenario feedback data cluster is not associated with the current reference feedback action type, the step of determining the current user feedback state in sequence in the ordered set of user feedback states based on the state sequence relationship is reversed to adjust the current user feedback state, and the step ends when the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type.

[0020] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the steps of analyzing the image application users associated with the feedback action type and the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application users corresponding to the user feedback state according to the feedback action type and the image application users associated with the reference feedback action type, include:

[0021] The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type;

[0022] Determining an action scene feedback data cluster corresponding to the reference feedback action type;

[0023] According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

[0024] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the user feedback states correspond to a state order relationship, and there is a positive correlation between the state order relationship and the quality of the user feedback states corresponding to the user feedback states;

[0025] The step of determining the candidate image application user corresponding to the user feedback state according to the action scene feedback data cluster corresponding to the reference feedback action type comprises:

[0026] Determine a current action scene feedback data cluster corresponding to a reference feedback action type corresponding to a current user feedback state, and merge the image application users in the current action scene feedback data cluster to form a current image application user cluster;

[0027] Determine a prior action scene feedback data cluster corresponding to a reference feedback action type corresponding to a prior user feedback state, and merge the image application users in the prior action scene feedback data cluster to form a prior image application user cluster, wherein the prior user feedback state is a user feedback state in the ordered set of user feedback states, whose state order relationship is prior to the current user feedback state;

[0028] In the current image application user cluster, the previous image application user cluster is eliminated to form a candidate image application user corresponding to the current user feedback status.

[0029] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the step of extracting an ordered set of user feedback states corresponding to the scene to be analyzed includes:

[0030] Determining a target action type change path, wherein the target action type change path includes a plurality of feedback action types sorted based on state depth;

[0031] Determining the action continuation probability corresponding to each feedback action type;

[0032] According to the action continuation probability corresponding to each of the feedback action types, the target action type change path is processed to form an ordered set of user feedback states corresponding to the scenario to be analyzed, and the reference feedback action type corresponding to the user feedback state is allocated to the feedback action type corresponding to the user feedback state.

[0033] In some preferred embodiments, in the above-mentioned AR-based information synthesis method, the step of determining the action continuation probability corresponding to each feedback action type includes:

[0034] Determine a typical to-be-analyzed scene feedback data cluster corresponding to the target action type change path, wherein the to-be-analyzed scene feedback data in the typical to-be-analyzed scene feedback data cluster includes feedback action types of typical application users on typical scene images, and the typical scene images belong to scene images corresponding to the typical scenes corresponding to the target action type change path;

[0035] Determining the number of typical application users of the same feedback action type in the typical scenario feedback data cluster to be analyzed, so as to output the number of typical users corresponding to the feedback action type;

[0036] Determining, according to the number of typical users corresponding to the feedback action type, the action continuation probability corresponding to the feedback action type;

[0037] The step of determining the action continuation probability corresponding to the feedback action type according to the number of typical users corresponding to the feedback action type includes:

[0038] Determine the current number of typical users corresponding to the current feedback action type;

[0039] Determine the number of typical users corresponding to the type of continuous feedback action;

[0040] The ratio between the number of continuing typical users and the number of current typical users is determined to obtain the action continuation probability corresponding to the current feedback action type.

[0041] The embodiment of the present invention further provides an AR-based information synthesis system, including:

[0042] A feedback data extraction module is used to extract a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed, wherein the scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes a type of feedback action of an image application user on an image of the scene to be analyzed, and the image of the scene to be analyzed belongs to a scene image corresponding to the scene to be analyzed;

[0043] A feedback state extraction module is used to extract an ordered set of user feedback states corresponding to the scene to be analyzed, wherein the ordered set of user feedback states includes a plurality of user feedback states sorted according to the merits of the user feedback states, and the ordered set of user feedback states is formed by processing based on the feedback action types corresponding to the action type change path of the scene to be analyzed;

[0044] An action type determination module, used to determine a reference feedback action type corresponding to a user feedback state in the ordered set of user feedback states;

[0045] an application user determination module, configured to analyze the image application users associated with the feedback action types and the reference feedback action types based on the scene feedback data cluster to be analyzed, and determine the candidate image application users corresponding to the user feedback state according to the image application users associated with the feedback action types and the reference feedback action types;

[0046] The application user screening module is used to determine a first user feedback state in the ordered set of user feedback states, and, among a plurality of candidate image application users corresponding to the first user feedback state, analyze the number of image application users corresponding to each preset user characterization information; and, based on the number of image application users corresponding to the preset user characterization information, determine the identifying user characterization information corresponding to the first user feedback state; and, among the candidate image application users corresponding to the second user feedback state, determine the candidate image application user having the identifying user characterization information and mark it as the target image application user, and the quality of the user feedback state corresponding to the second user feedback state is worse than that of the user feedback state corresponding to the first user feedback state;

[0047] The image synthesis module is used to merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene synthetic image.

[0048] In some preferred embodiments, in the above-mentioned AR-based information synthesis system, the application user determination module is specifically used to:

[0049] In the scene feedback data cluster to be analyzed, determining the scene feedback data to be analyzed corresponding to each image application user to form a user scene feedback data cluster corresponding to the image application user;

[0050] In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

[0051] In some preferred embodiments, in the above-mentioned AR-based information synthesis system, the application user determination module is specifically used to:

[0052] The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type;

[0053] Determining an action scene feedback data cluster corresponding to the reference feedback action type;

[0054] According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

[0055] An AR-based information synthesis method and system provided by an embodiment of the present invention can first extract a scene feedback data cluster to be analyzed corresponding to the scene to be analyzed; extract an ordered set of user feedback states corresponding to the scene to be analyzed; determine a reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states; analyze image application users associated with the feedback action type and the reference feedback action type based on the scene feedback data cluster to be analyzed, and determine candidate image application users corresponding to the user feedback state according to the image application users associated with the feedback action type and the reference feedback action type; determine a first user feedback state in the ordered set of user feedback states, and analyze the number of image application users corresponding to each preset user representation information among multiple candidate image application users corresponding to the first user feedback state; and determine the identifying user representation information corresponding to the first user feedback state according to the number of image application users corresponding to the preset user representation information; and determine the candidate image application users with the identifying user representation information among the candidate image application users corresponding to the second user feedback state, and mark them as target image application users; merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user scene synthetic image. Based on the above method, the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user can be made to match more closely, thereby improving the reliability of information synthesis and improving the deficiencies in the prior art.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a structural block diagram of an AR-based information synthesis platform provided in an embodiment of the present invention.

[0058] Figure 2 A flowchart of the steps of the AR-based information synthesis method provided in an embodiment of the present invention.

[0059] Figure 3 A schematic diagram of various modules included in the AR-based information synthesis system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] like Figure 1 As shown, an embodiment of the present invention provides an AR-based information synthesis platform. The AR-based information synthesis platform may include a memory and a processor.

[0063] In detail, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The memory may store at least one software function module (computer program) that may exist in the form of software or firmware. The processor may be used to execute the executable computer program stored in the memory, thereby implementing the AR-based information synthesis method provided by the embodiment of the present invention (as described later).

[0064] It can be selected that in some embodiments, the memory can be, but not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electric erasable programmable read-only memory (EEPROM), etc. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0065] Optionally, in some implementations, the AR-based information synthesis platform may be a server with data processing capabilities.

[0066] Combination Figure 2 The embodiment of the present invention further provides an AR-based information synthesis method, which can be applied to the above-mentioned AR-based information synthesis platform. Among them, the method steps defined in the process related to the AR-based information synthesis method can be implemented by the AR-based information synthesis platform.

[0067] The following will Figure 2 The specific process shown is explained in detail.

[0068] Step S110: extracting a scene feedback data cluster to be analyzed corresponding to the scene to be analyzed.

[0069] In an embodiment of the present invention, the AR-based information synthesis platform can extract a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed. The scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes the type of feedback action of the image application user on the scene image to be analyzed, and the scene image to be analyzed belongs to the scene image corresponding to the scene to be analyzed. The feedback action type may refer to the processing action on the scene image to be analyzed, such as being divided into receiving the scene image to be analyzed, viewing the scene image to be analyzed, editing the scene image to be analyzed, sending the scene image to be analyzed, etc. in order of the depth of processing from small to large.

[0070] Step S120: extracting an ordered set of user feedback states corresponding to the scenario to be analyzed.

[0071] In an embodiment of the present invention, the AR-based information synthesis platform can extract an ordered set of user feedback states corresponding to the scene to be analyzed. The ordered set of user feedback states includes multiple user feedback states sorted according to the pros and cons of the user feedback states, and the ordered set of user feedback states is formed by processing the feedback action types corresponding to the action type change path of the scene to be analyzed. In addition, the feedback action types in the action type change path can be sorted according to the corresponding processing depth, such as in order from small to large depth.

[0072] Step S130: determining a reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states.

[0073] In an embodiment of the present invention, the AR-based information synthesis platform can determine the reference feedback action types corresponding to the user feedback states in the ordered set of user feedback states. The number of the reference feedback action types can be greater than or equal to 1, that is, more than one.

[0074] Step S140, based on the scene feedback data cluster to be analyzed, analyzing the feedback action type and the image application users associated with the reference feedback action type, and, according to the feedback action type and the image application users associated with the reference feedback action type, determining the candidate image application users corresponding to the user feedback status.

[0075] In an embodiment of the present invention, the AR-based information synthesis platform can analyze the image application users whose feedback action types are associated with the reference feedback action types based on the scene feedback data cluster to be analyzed, and determine the candidate image application users corresponding to the user feedback state according to the image application users whose feedback action types are associated with the reference feedback action types. The term "associated" can mean having consistency, and thus can be determined by one-to-one comparison.

[0076] Step S150, determining a first user feedback state in the ordered set of user feedback states, and analyzing the number of image application users corresponding to each preset user characterization information among multiple candidate image application users corresponding to the first user feedback state; and determining the identifying user characterization information corresponding to the first user feedback state based on the number of image application users corresponding to the preset user characterization information; and determining the candidate image application user having the identifying user characterization information among the candidate image application users corresponding to the second user feedback state, and marking the candidate image application user as the target image application user.

[0077] In an embodiment of the present invention, the AR-based information synthesis platform can determine a first user feedback state in the ordered set of user feedback states, and analyze the number of image application users corresponding to each preset user characterization information among multiple candidate image application users corresponding to the first user feedback state; and determine the identifying user characterization information corresponding to the first user feedback state according to the number of image application users corresponding to the preset user characterization information; and, among the candidate image application users corresponding to the second user feedback state, determine the candidate image application user with the identifying user characterization information and mark it as the target image application user. The user feedback state corresponding to the second user feedback state is inferior to the user feedback state corresponding to the first user feedback state, such as the processing progress is shallower. In addition, the first user feedback state can be any user feedback state in the ordered set of user feedback states, and the first user feedback state can be determined from the ordered set of user feedback states according to actual needs. The preset user characterization information can be determined according to actual needs, and the preset user characterization information can be used to characterize the preference characteristics of image application users for different scenes, etc. In addition, from the number of image application users corresponding to each preset user characterization information, a number of image application users greater than the preset number of users can be selected as the candidate number of image application users, and the preset user characterization information corresponding to the candidate number of image application users can be used as the identifying user characterization information.

[0078] Step S160 , merging the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene composite image.

[0079] In an embodiment of the present invention, the AR-based information synthesis platform can merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user scene synthesis image. For example, the scene image can be used as the background part, and the user image can be used as the foreground part.

[0080] Based on the above method, the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user can be made to match more closely, thereby improving the reliability of information synthesis and improving the deficiencies in the prior art (such as arbitrary combinations between scene images and user images).

[0081] Optionally, in some implementations, the steps of analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type, may further include the following contents:

[0082] In the scene feedback data cluster to be analyzed, the scene feedback data to be analyzed corresponding to each image application user is determined to form a user scene feedback data cluster corresponding to the image application user, that is, a user scene feedback data cluster corresponding to each image application user is formed respectively;

[0083] In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

[0084] Alternatively, in some implementations, the user feedback states correspond to a state sequence relationship, such as the aforementioned processing depth, and there is a positive correlation between the state sequence relationship and the quality of the user feedback states corresponding to the user feedback states, such as the shallower the processing depth, the worse the quality. Based on this, the step of determining the reference feedback action type corresponding to the user feedback states in the ordered set of user feedback states may further include the following content:

[0085] Based on the state sequence relationship, the current user feedback state is determined in sequence in the ordered set of user feedback states, that is, it is sequentially used as the current user feedback state, and the current reference feedback action type corresponding to the current user feedback state is determined.

[0086] Based on the above content, when the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, the step of marking the image application user as a candidate image application user corresponding to the user feedback state includes:

[0087] In a case where the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type, performing a marking operation on the image application user to mark the image application user as a candidate image application user corresponding to the current user feedback state;

[0088] In the case where the feedback action type in the user scenario feedback data cluster is not associated with the current reference feedback action type, the step of determining the current user feedback state in sequence in the ordered set of user feedback states based on the state sequence relationship is reversed to adjust the current user feedback state, that is, update is achieved, and the process ends when the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type.

[0089] Optionally, in some implementations, the steps of analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type, may further include the following contents:

[0090] The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type, that is, the scene feedback data to be analyzed in one action scene feedback data cluster corresponds to the same feedback action type;

[0091] Determining an action scene feedback data cluster corresponding to the reference feedback action type;

[0092] According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

[0093] Optionally, in some implementations, the user feedback states correspond to a state sequence relationship, and there is a positive correlation between the state sequence relationship and the quality of the user feedback states corresponding to the user feedback states. Based on this, the step of determining the candidate image application user corresponding to the user feedback state according to the action scene feedback data cluster corresponding to the reference feedback action type may further include the following content:

[0094] Determine a current action scene feedback data cluster corresponding to a reference feedback action type corresponding to a current user feedback state, and merge the image application users in the current action scene feedback data cluster to form a current image application user cluster;

[0095] Determine a prior action scene feedback data cluster corresponding to a reference feedback action type corresponding to a prior user feedback state, and merge the image application users in the prior action scene feedback data cluster to form a prior image application user cluster, wherein the prior user feedback state is a user feedback state in the ordered set of user feedback states, whose state order relationship is prior to the current user feedback state, such as a shallower processing depth;

[0096] In the current image application user cluster, the previous image application user cluster is eliminated to form a candidate image application user corresponding to the current user feedback status.

[0097] Optionally, in some implementations, the step of extracting an ordered set of user feedback states corresponding to the scene to be analyzed may further include the following content:

[0098] Determine a target action type change path, wherein the target action type change path includes a plurality of feedback action types sorted based on state depth, the state depth being the depth of the aforementioned processing, and the target action type change path is an action type change path corresponding to the scene to be analyzed;

[0099] Determine an action continuation probability corresponding to each feedback action type, wherein the action continuation probability is used to reflect the probability of the image application user who has generated the feedback action type continuing to generate the next feedback action type after the processing action corresponding to the feedback action type has generated;

[0100] According to the action continuation probability corresponding to each of the feedback action types, the target action type change path is processed to form an ordered set of user feedback states corresponding to the scenario to be analyzed, and the reference feedback action type corresponding to the user feedback state is allocated to the feedback action type corresponding to the user feedback state.

[0101] Optionally, in some implementations, the step of determining the action continuation probability corresponding to each feedback action type may further include the following content:

[0102] Determine a typical to-be-analyzed scene feedback data cluster corresponding to the target action type change path, wherein the to-be-analyzed scene feedback data in the typical to-be-analyzed scene feedback data cluster includes feedback action types of typical application users on typical scene images, and the typical scene images belong to scene images corresponding to the typical scenes corresponding to the target action type change path;

[0103] Determining the number of typical application users of the same feedback action type in the typical scenario feedback data cluster to be analyzed, so as to output the number of typical users corresponding to the feedback action type;

[0104] According to the number of typical users corresponding to the feedback action type, the action continuation probability corresponding to the feedback action type is determined.

[0105] Wherein, it may be selected that, in some implementations, the step of determining the action continuation probability corresponding to the feedback action type according to the number of typical users corresponding to the feedback action type may further include the following content:

[0106] Determine the current number of typical users corresponding to the current feedback action type;

[0107] Determining the number of continuing typical users corresponding to the continuing feedback action type, wherein the continuing feedback action type may be the next feedback action type with a deeper processing depth;

[0108] The ratio between the number of continuing typical users and the number of current typical users is determined to obtain the action continuation probability corresponding to the current feedback action type.

[0109] Optionally, in some implementations, the step of processing the target action type change path according to the action continuation probability corresponding to each feedback action type to form an ordered set of user feedback states corresponding to the scene to be analyzed may further include the following content:

[0110] Determine, according to the action continuation probability corresponding to each feedback action type, the probability difference between the action continuation probabilities of adjacent feedback action types in the target action type change path;

[0111] Adjacent feedback action types whose probability difference is less than or equal to the reference probability difference are assigned to the same user feedback state. For example, the action continuation probability of the first feedback action type can be obtained as the first action continuation probability, and the action continuation probability of the second feedback action type can be obtained as the second action continuation probability. The first action continuation probability and the second action continuation probability are subjected to difference operation, and the result of the difference operation is used as the probability difference. The probability difference is compared with the reference probability difference. When the probability difference is less than the reference probability difference, the first feedback action type and the second feedback action type are classified into the same user feedback state, otherwise, the first feedback action type and the second feedback action type are classified into different user feedback states. The first feedback action type and the second feedback action type are adjacent feedback action types in the target action type change path.

[0112] Combination Figure 3 The embodiment of the present invention further provides an AR-based information synthesis system, which can be applied to the above-mentioned AR-based information synthesis platform. The AR-based information synthesis system includes:

[0113] A feedback data extraction module is used to extract a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed, wherein the scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes a type of feedback action of an image application user on an image of the scene to be analyzed, and the image of the scene to be analyzed belongs to a scene image corresponding to the scene to be analyzed;

[0114] A feedback state extraction module is used to extract an ordered set of user feedback states corresponding to the scene to be analyzed, wherein the ordered set of user feedback states includes a plurality of user feedback states sorted according to the merits of the user feedback states, and the ordered set of user feedback states is formed by processing based on the feedback action types corresponding to the action type change path of the scene to be analyzed;

[0115] An action type determination module, used to determine a reference feedback action type corresponding to a user feedback state in the ordered set of user feedback states;

[0116] an application user determination module, configured to analyze the image application users associated with the feedback action types and the reference feedback action types based on the scene feedback data cluster to be analyzed, and determine the candidate image application users corresponding to the user feedback state according to the image application users associated with the feedback action types and the reference feedback action types;

[0117] The application user screening module is used to determine a first user feedback state in the ordered set of user feedback states, and, among a plurality of candidate image application users corresponding to the first user feedback state, analyze the number of image application users corresponding to each preset user characterization information; and, based on the number of image application users corresponding to the preset user characterization information, determine the identifying user characterization information corresponding to the first user feedback state; and, among the candidate image application users corresponding to the second user feedback state, determine the candidate image application user having the identifying user characterization information and mark it as the target image application user, and the quality of the user feedback state corresponding to the second user feedback state is worse than that of the user feedback state corresponding to the first user feedback state;

[0118] The image synthesis module is used to merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene synthetic image.

[0119] Optionally, in some implementations, the application user determination module is specifically used to:

[0120] In the scene feedback data cluster to be analyzed, determining the scene feedback data to be analyzed corresponding to each image application user to form a user scene feedback data cluster corresponding to the image application user;

[0121] In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

[0122] Optionally, in some implementations, the application user determination module is specifically used to:

[0123] The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type;

[0124] Determining an action scene feedback data cluster corresponding to the reference feedback action type;

[0125] According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

[0126] In summary, the AR-based information synthesis method and system provided by the present invention can first extract a scene feedback data cluster to be analyzed corresponding to the scene to be analyzed; extract an ordered set of user feedback states corresponding to the scene to be analyzed; determine the reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states; analyze the image application users associated with the feedback action type and the reference feedback action type based on the scene feedback data cluster to be analyzed, and determine the candidate image application users corresponding to the user feedback state according to the image application users associated with the feedback action type and the reference feedback action type; determine the first user feedback state in the ordered set of user feedback states, and analyze the number of image application users corresponding to each preset user representation information among multiple candidate image application users corresponding to the first user feedback state; and determine the identification user representation information corresponding to the first user feedback state according to the number of image application users corresponding to the preset user representation information; and determine the candidate image application users with the identification user representation information among the candidate image application users corresponding to the second user feedback state, and mark them as target image application users; merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user scene synthetic image. Based on the above method, the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user can be made to match more closely. Therefore, the reliability of information synthesis can be improved and the deficiencies in the prior art (such as the problem of low reliability caused by any combination of scene images and user images) can be improved.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An information synthesis method based on AR, characterized in that: include: Extracting a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed, wherein the scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes a type of feedback action of an image application user on an image of the scene to be analyzed, and the image of the scene to be analyzed belongs to a scene image corresponding to the scene to be analyzed; Extracting an ordered set of user feedback states corresponding to the scene to be analyzed, the ordered set of user feedback states comprising a plurality of user feedback states sorted according to the merits of the user feedback states, the ordered set of user feedback states being formed by processing based on the feedback action types corresponding to the action type change paths of the scene to be analyzed, wherein the shallower the processing depth, the worse the merits, and the ordered set of user feedback states can be sequentially divided into receiving an image of the scene to be analyzed, viewing the image of the scene to be analyzed, editing the image of the scene to be analyzed, and sending the image of the scene to be analyzed in order from small to large processing depths; Determining a reference feedback action type corresponding to a user feedback state in the ordered set of user feedback states; Analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type; In the ordered set of user feedback states, a first user feedback state is determined, and, among a plurality of candidate image application users corresponding to the first user feedback state, the number of image application users corresponding to each preset user characterization information is analyzed; and, according to the number of image application users corresponding to the preset user characterization information, the identifying user characterization information corresponding to the first user feedback state is determined; and, among the candidate image application users corresponding to the second user feedback state, a candidate image application user having the identifying user characterization information is determined and marked as a target image application user, and the quality of the user feedback state corresponding to the second user feedback state is worse than that of the user feedback state corresponding to the first user feedback state; A merging operation is performed on the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene composite image.

2. The AR-based information synthesis method according to claim 1, characterized in that: The steps of analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type, include: In the scene feedback data cluster to be analyzed, determining the scene feedback data to be analyzed corresponding to each image application user to form a user scene feedback data cluster corresponding to the image application user; In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

3. The AR-based information synthesis method according to claim 2, characterized in that: The user feedback states correspond to a state order relationship, and there is a positive correlation between the state order relationship and the quality of the user feedback states corresponding to the user feedback states; The step of determining the reference feedback action type corresponding to the user feedback state in the ordered set of user feedback states comprises: determining the current user feedback state in the ordered set of user feedback states in order based on the state sequence relationship, and determining the current reference feedback action type corresponding to the current user feedback state; When the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, the step of marking the image application user as a candidate image application user corresponding to the user feedback state comprises: In a case where the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type, performing a marking operation on the image application user to mark the image application user as a candidate image application user corresponding to the current user feedback state; When the feedback action type in the user scenario feedback data cluster is not associated with the current reference feedback action type, the step of determining the current user feedback state in sequence in the ordered set of user feedback states based on the state sequence relationship is reversed to adjust the current user feedback state, and the step ends when the feedback action type in the user scenario feedback data cluster is associated with the current reference feedback action type.

4. The AR-based information synthesis method according to claim 1, characterized in that: The steps of analyzing the feedback action type and the image application user associated with the reference feedback action type based on the scene feedback data cluster to be analyzed, and determining the candidate image application user corresponding to the user feedback state according to the feedback action type and the image application user associated with the reference feedback action type, include: The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type; Determining an action scene feedback data cluster corresponding to the reference feedback action type; According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

5. The AR-based information synthesis method according to claim 4, characterized in that: The user feedback states correspond to a state order relationship, and there is a positive correlation between the state order relationship and the quality of the user feedback states corresponding to the user feedback states; The step of determining the candidate image application user corresponding to the user feedback state according to the action scene feedback data cluster corresponding to the reference feedback action type comprises: Determine a current action scene feedback data cluster corresponding to a reference feedback action type corresponding to a current user feedback state, and merge the image application users in the current action scene feedback data cluster to form a current image application user cluster; Determine a prior action scene feedback data cluster corresponding to a reference feedback action type corresponding to a prior user feedback state, and merge the image application users in the prior action scene feedback data cluster to form a prior image application user cluster, wherein the prior user feedback state is a user feedback state in the ordered set of user feedback states, whose state order relationship is prior to the current user feedback state; In the current image application user cluster, the previous image application user cluster is eliminated to form a candidate image application user corresponding to the current user feedback status.

6. The AR-based information synthesis method according to claim 1, characterized in that: The step of extracting an ordered set of user feedback states corresponding to the scene to be analyzed includes: Determine a target action type change path, wherein the target action type change path includes a plurality of feedback action types sorted based on state depth, wherein the state depth is the depth of the aforementioned processing; Determining the action continuation probability corresponding to each feedback action type; According to the action continuation probability corresponding to each of the feedback action types, the target action type change path is processed to form an ordered set of user feedback states corresponding to the scenario to be analyzed, and the reference feedback action type corresponding to the user feedback state is allocated to the feedback action type corresponding to the user feedback state.

7. The AR-based information synthesis method according to claim 6, characterized in that: The step of determining the action continuation probability corresponding to each feedback action type includes: Determine a typical to-be-analyzed scene feedback data cluster corresponding to the target action type change path, wherein the to-be-analyzed scene feedback data in the typical to-be-analyzed scene feedback data cluster includes feedback action types of typical application users on typical scene images, and the typical scene images belong to scene images corresponding to the typical scenes corresponding to the target action type change path; Determining the number of typical application users of the same feedback action type in the typical scenario feedback data cluster to be analyzed, so as to output the number of typical users corresponding to the feedback action type; Determining, according to the number of typical users corresponding to the feedback action type, an action continuation probability corresponding to the feedback action type; The step of determining the action continuation probability corresponding to the feedback action type according to the number of typical users corresponding to the feedback action type includes: Determine the current number of typical users corresponding to the current feedback action type; Determine the number of typical users corresponding to the type of feedback action; The ratio between the number of continuing typical users and the number of current typical users is determined to obtain the action continuation probability corresponding to the current feedback action type.

8. An AR-based information synthesis system, characterized in that: include: A feedback data extraction module is used to extract a scene to be analyzed feedback data cluster corresponding to the scene to be analyzed, wherein the scene to be analyzed feedback data in the scene to be analyzed feedback data cluster includes a type of feedback action of an image application user on an image of the scene to be analyzed, and the image of the scene to be analyzed belongs to a scene image corresponding to the scene to be analyzed; A feedback state extraction module is used to extract an ordered set of user feedback states corresponding to the scene to be analyzed, wherein the ordered set of user feedback states includes a plurality of user feedback states sorted according to the quality of the user feedback states, and the ordered set of user feedback states is formed by processing the feedback action types corresponding to the action type change paths of the scene to be analyzed, wherein the shallower the processing depth, the worse the quality, and the order of processing depth from small to large can be sequentially divided into receiving the scene image to be analyzed, viewing the scene image to be analyzed, editing the scene image to be analyzed, and sending the scene image to be analyzed; An action type determination module, used to determine a reference feedback action type corresponding to a user feedback state in the ordered set of user feedback states; an application user determination module, configured to analyze the image application users associated with the feedback action types and the reference feedback action types based on the scene feedback data cluster to be analyzed, and determine the candidate image application users corresponding to the user feedback state according to the image application users associated with the feedback action types and the reference feedback action types; The application user screening module is used to determine a first user feedback state in the ordered set of user feedback states, and, among a plurality of candidate image application users corresponding to the first user feedback state, analyze the number of image application users corresponding to each preset user characterization information; and, based on the number of image application users corresponding to the preset user characterization information, determine the identifying user characterization information corresponding to the first user feedback state; and, among the candidate image application users corresponding to the second user feedback state, determine the candidate image application user having the identifying user characterization information and mark it as the target image application user, and the quality of the user feedback state corresponding to the second user feedback state is worse than that of the user feedback state corresponding to the first user feedback state; The image synthesis module is used to merge the scene image corresponding to the scene to be analyzed and the user image corresponding to the target image application user to form a corresponding user-scene synthetic image.

9. The AR-based information synthesis system according to claim 8, characterized in that: The application user determination module is specifically used for: In the scene feedback data cluster to be analyzed, determining the scene feedback data to be analyzed corresponding to each image application user to form a user scene feedback data cluster corresponding to the image application user; In a case where the feedback action type in the user scenario feedback data cluster is associated with the reference feedback action type, a marking operation is performed on the image application user to mark the image application user as a candidate image application user corresponding to the user feedback state.

10. The AR-based information synthesis system according to claim 8, characterized in that: The application user determination module is specifically used for: The scene feedback data clusters to be analyzed are grouped based on the feedback action types to form action scene feedback data clusters corresponding to each feedback action type; Determining an action scene feedback data cluster corresponding to the reference feedback action type; According to the action scene feedback data cluster corresponding to the reference feedback action type, a candidate image application user corresponding to the user feedback state is determined.

Images