Digital twin display method, system, device and medium for intelligent identification

Abstract

The present invention relates to the technical field of intelligent sign display, and in particular to a digital twin display method, system, device and medium for intelligent sign, including obtaining user position and posture information, calling corresponding device to identify user identity based on the user position and posture information, and obtaining user identity information; obtaining available device information based on the user identity information and the user position and posture information; collecting user behavior information based on the available device information; inputting the user identity information and the user behavior information into a machine learning model, obtaining user habit information and device control information, the user device control is used to optimize the interaction mode between different devices and users according to user habits; controlling the corresponding device based on the device control information, collecting user reaction behavior information, inputting the user reaction behavior information into the machine learning model, and training the machine learning model through the user behavior information and the user reaction behavior information. The present application has the effect of improving the immersion, participation and interactive experience of the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of smart sign display, and in particular to a digital twin display method, system, device and medium for smart signs. Background Art

[0002] Traditional exhibition display technology focuses on static physical display, which is difficult to meet the needs of modern audiences for immersion and interactivity. In recent years, with the development of the Internet of Things, artificial intelligence and big data technologies, virtual reality and digital twin technologies have gradually become an effective way to enhance the exhibition experience.

[0003] Currently common display systems usually rely on a single sensor to locate users, lacking high-precision positioning capabilities and multi-modal interaction support, resulting in low integration between virtual elements and the real environment and limited user experience.

[0004] However, the existing display system is insufficient in terms of multi-terminal access, real-time feedback and personalized content push, especially the inability to effectively process large amounts of sensor data, resulting in response delays and decreased data accuracy, reducing the realism and immersion of the display effect. Therefore, there is room for improvement. Summary of the invention

[0005] In order to improve the immersion, participation and interactive experience of the user, the present application provides a digital twin display method, system, device and medium for smart signs.

[0006] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A digital twin display method for a smart logo, the digital twin display method for a smart logo comprising the steps of: Acquire user location and posture information, and based on the user location and posture information, call a corresponding device to identify the user and obtain user identity information; Acquire available device information according to the user identity information and the user location and posture information; Based on the available device information, collect user behavior information; Inputting the user identity information and the user behavior information into a machine learning model to obtain user habit information and device control information, wherein the user device control is used to optimize the interaction mode between different devices and users according to user habits; Based on the device control information, the corresponding device is controlled, user reaction behavior information is collected, the user reaction behavior information is input into the machine learning model, and the machine learning model is trained through the user behavior information and the user reaction behavior information.

[0007] By adopting the above technical solution, during the use of the smart display device, when calling the device connected to the smart display device or running the service program of the smart display device, the device with user identity recognition in the device connected to the smart display device is used to identify the user, so as to quickly determine the identity of the user, facilitate the subsequent collection of user behavior or device control, ensure the response speed of the smart display device, and thus improve the user's usage experience; after successfully identifying the user's identity, the current appropriate device for collecting user behavior will be determined based on the specific business type handled by the business. For example, when the user handles an account query business, the user is required to enter the account information. At this time, the device related to the business handled by the user has an ID reading device. Therefore, at this time, User behavior is collected through an unnecessary camera device, and the camera is not offensive to the user, that is, the camera is not parallel to the user's eyes, so as to ensure that the device collecting user behavior information will not reduce the user experience; in the machine learning model, the behavior of each user when handling each business is learned, and then the user's satisfaction with different business handling is judged, and, based on the behavior of each user when handling each business, the business handling process suitable for the user's habits is inferred, so as to implement a more suitable service process for the user; user behavior is collected every time the user handles the business, and the machine learning model is trained to continuously improve the accuracy of judging user habits, thereby generating a service process that is more suitable for user habits and improving user experience satisfaction.

[0008] In a preferred example, the present application can be further configured as follows: based on the business handling information, calling the corresponding device to identify the user identity and obtaining the user identity information specifically includes: Based on the business handling information, the corresponding device is called to identify the user's identity and determine whether the user enters the identity authentication information; If the judgment is yes, the identity verification information is compared with the user identity recognized by the device, and according to the comparison result, an identity verification inconsistency instruction is triggered or the user identity information is obtained; If the judgment is no, the user identity information is obtained according to the user identity recognized by the device.

[0009] By adopting the above technical solution, in the application scenarios of smart display devices, some services may not require users to enter identity authentication information, for example, services that allow users to query the business hall's available services or services that only require the use of staff's authority to query information, while some services require users to handle them in person. In addition, there are also services that do not require users to handle them in person. For services that require users to handle them in person but do not require users to enter identity authentication information, staff generally manually verify the user's identity. Therefore, in order to ensure the standardization of business processes and further ensure the accuracy of identifying user identities, the user identity identified by the device is compared with the identity authentication information entered by the user. When the comparison is inconsistent, the staff will be reminded to achieve security protection of user information, thereby improving users' trust and satisfaction with smart display devices.

[0010] In a preferred example, the present application may be further configured as follows: obtaining available device information according to the user identity information and the business handling information specifically includes: According to the business handling information, obtain device locking information; According to the user identity information, obtain corresponding user device locking information; Based on the device locking information and the user device locking information, available device information is acquired.

[0011] By adopting the above technical solution, the device lock information refers to the device that will be called by the user's current business. In order to ensure the smooth progress of the user's business handling, the device that will be called by the user's current business is not used to collect user information, but user behavior information is collected through other idle devices. In addition, the user device lock information refers to the device that is not used to collect user behavior information based on each user's habits. For example, the historically collected user behavior information indicates that when the camera device is used to collect the user's information, the user's behavior information shows resistance, then the user is marked with the label information of the camera device locking, and when the user is subsequently identified as the user, the camera device will no longer be used to collect user information. Based on this, targeted behavior collection of different users is achieved under the premise of ensuring the user's usage experience.

[0012] In a preferred example, the present application may be further configured as follows: collecting user behavior information based on the available device information specifically includes: Based on the available device information and the user identity information, acquiring device behavior control information; Based on the device behavior control information, the corresponding device is controlled to collect user behavior information.

[0013] By adopting the above technical solution, device behavior control information refers to the control information of the device in the process of collecting user behavior information, for example, controlling the user behavior collection frequency of the device and controlling the operating power of the device, or performing linkage control on some devices to obtain more accurate user behavior information. Therefore, by using the device behavior control information corresponding to the business type handled by each user, the accuracy of the collection of user behavior information of the user is guaranteed, the user experience is guaranteed, and the probability of user resistance is reduced.

[0014] In a preferred example, the present application may be further configured as follows: the user identity information and the user behavior information are input into a machine learning model to obtain user habit information and device control information, specifically including: Input the user identity information and the user behavior information into a machine learning model to extract corresponding user habit information and user expectation ranking information, where the user expectation ranking information is used to represent the user's demand for the business handling process; Device control information is acquired according to the user habit information and the user desired sorting information.

[0015] By adopting the above technical solution, the machine learning model is a deep learning model that has the function of identifying user behavior, and can understand the meaning of user behavior and learn user habits through multiple user behavior information. Therefore, after the user behavior information is input into the machine learning model, the user's behavioral habits when handling the current business are judged, as well as the user's needs for the current business handling process. For example, the user hopes that the speed of business handling will be increased or the user hopes that the waiting time during the business handling process will be reduced. Therefore, through the user habit information and the user expectation sorting information, the device control information that conforms to the user's habits and meets the user's expectations is comprehensively judged. For example, more computing resources are called to process the current business to increase the business handling speed, or by changing the data channel between the smart display device and the customer management system, the data interaction speed between the smart display device and the customer management system is increased, thereby increasing the business handling speed of the smart display device. Based on this, a more suitable service process is implemented for the user, thereby improving the user experience satisfaction.

[0016] In a preferred example, the present application may be further configured as follows: acquiring device control information according to the user's desired sorting information specifically includes: The user expected sorting information sorts the service processing speed, service waiting time, service processing flow visibility and service processing result display visibility. The service processing speed, service waiting time, service processing flow visibility and service processing result display visibility all correspond to a plurality of equipment control information with different optimization levels. The user expected sorting information is used to control the optimization levels of different equipment. According to the user's expected sorting information, device control information of a corresponding optimization degree is obtained.

[0017] By adopting the above technical solution, the user expectation sorting information will sort the business processing speed, business waiting time, business processing process visibility and business processing result display visibility. The business waiting time refers to the time the user waits during the business processing process, rather than the time the user waits for the business to be processed. The business processing process visibility refers to the visibility of the specific next step guidance when the user processes the business. The business processing result display visibility refers to the visibility of the form of displaying the business processing result. Different users have different perceptions of business processing speed, business waiting time, business processing process visibility and business processing result display visibility, and their corresponding expectations are also different. Therefore, according to the user's expected ranking of business processing speed, business waiting time, business processing process visibility and business processing result display visibility, the control device is optimized with the corresponding ranking to avoid excessive optimization and counterproductive effects. Corresponding optimization is performed in the aspects that users expect more, which can further improve user satisfaction, make more reasonable use of the resources of smart display devices, and extend the service life of smart display devices.

[0018] In the second aspect, the above invention objective of the present application is achieved through the following technical solutions: A digital twin display device of a smart sign, the digital twin display device of the smart sign comprising: The user identity recognition module is used to call the corresponding device to identify the user's identity and obtain the user's identity information based on the business handling information; An available device acquisition module, used to acquire available device information according to the user identity information and the service handling information; A user behavior collection module, used to collect user behavior information based on the available device information; A model output module, used to input the user identity information and the user behavior information into a machine learning model to obtain user habit information and device control information, wherein the user device control is used to optimize the interaction mode between different devices and users according to user habits; A model training module is used to control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

[0019] Optionally, the user identity recognition module includes: The user input judgment submodule is used to call the corresponding device to identify the user's identity based on the business handling information and determine whether the user enters the identity authentication information; The identity comparison submodule is used to compare the identity verification information with the user identity recognized by the device if the judgment is yes, and trigger an identity verification inconsistency instruction or obtain the user identity information according to the comparison result; The user identity acquisition submodule is used to obtain user identity information according to the user identity recognized by the device if the judgment is no.

[0020] In the third aspect, the above invention objective of the present application is achieved through the following technical solutions: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the digital twin display method of the above-mentioned smart logo are implemented.

[0021] In a fourth aspect, the above-mentioned invention objective of the present application is achieved through the following technical solutions: A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the digital twin display method of the above-mentioned smart logo.

[0022] In summary, the present application includes at least one of the following beneficial technical effects: 1. During the use of the smart display device, when calling the device connected to the smart display device or running the service program of the smart display device, the device with user identity recognition in the device connected to the smart display device is used to identify the user, so as to quickly determine the identity of the user, facilitate the subsequent collection of user behavior or device control, ensure the response speed of the smart display device, and thus improve the user's usage experience; after successfully identifying the user's identity, the current appropriate device for collecting user behavior will be determined based on the specific business type handled by the business. For example, when the user handles an account query business, the user is required to enter the account information. At this time, the device related to the business handled by the user has a certificate reading device. Therefore, at this time, it can be done without The camera device to be used collects user behavior, and the camera is not offensive to the user, that is, it is not parallel to the user's eyes, so as to ensure that the device collecting user behavior information will not reduce the user's experience; in the machine learning model, the behavior of each user when handling each business is learned, and then the user's satisfaction with different business handling is judged, and based on the behavior of each user when handling each business, the business handling process suitable for the user's habits is inferred, so as to implement a more suitable service process for the user; user behavior is collected every time the user handles the business, and the machine learning model is trained to continuously improve the accuracy of judging user habits, so as to generate a service process that is more suitable for user habits and improve user experience satisfaction; 2. In the application scenarios of smart display devices, some services may not require users to enter identity authentication information, for example, to query the services that can be handled in the business hall for users or to query information with the staff's authority, while some services need to be handled by the users themselves. In addition, there are also services that do not require users to handle them themselves. For services that require users to handle them themselves but do not require users to enter identity authentication information, the staff generally manually verifies the user's identity. Therefore, in order to ensure the standardization of business processes and further ensure the accuracy of identifying user identities, the user identity identified by the device is compared with the identity authentication information entered by the user. When the comparison is inconsistent, the staff will be reminded to achieve security protection of user information, thereby improving users' trust and satisfaction with smart display devices; 3. Device lock information refers to the device that will be called for the business that the user is currently handling. In order to ensure the smooth progress of the user's business handling, the device that will be called for the business that the user is currently handling is not used to collect user information, but user behavior information is collected through other idle devices. In addition, user device lock information refers to the device that is not used to collect user behavior information based on each user's habits. For example, historically collected user behavior information indicates that when a camera is used to collect user information, the user behavior information shows resistance, then the user is marked with a label information that the camera is locked, and when the user is subsequently identified as this user, the camera will no longer be used to collect user information. Based on this, targeted behavior collection of different users is achieved under the premise of ensuring the user's usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flowchart of an implementation method of a digital twin display method of a smart sign in an embodiment of the present application; Figure 2 It is a flowchart for implementing S10 of the digital twin display method of the smart logo in the embodiment of the present application; Figure 3 It is a flowchart for implementing S20 of the digital twin display method of the smart logo in the embodiment of the present application; Figure 4 It is a flowchart for implementing S30 of the digital twin display method of the smart logo in the embodiment of the present application; Figure 5 It is a flowchart for implementing S40 of the digital twin display method of the smart logo in the embodiment of the present application; Figure 6 It is a flowchart for implementing S42 of the digital twin display method of the smart logo in the embodiment of the present application; Figure 7 It is a principle block diagram of a digital twin display device of a smart sign in an embodiment of the present application; Figure 8It is a diagram of the internal structure of a digital twin display computer device of a smart logo in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following is combined with Figure 1-8 This application is described in further detail.

[0025] In one embodiment, if Figure 1 As shown, the present application discloses a digital twin display method of a smart logo, which specifically includes the following steps: S10: Based on the business handling information, the corresponding device is called to identify the user identity and obtain the user identity information.

[0026] In this embodiment, the smart display device is used at the service window of the telecommunications business hall. The smart display device is connected to the user management system of the telecommunications enterprise. The devices connected to the smart display device may include high-speed scanners, monocular / binocular portrait cameras, multi-document recognition devices, card reading and writing devices, and electromagnetic signature devices. The smart display device can be connected to noise reduction recording devices, code scanning payment devices, fingerprint recognition devices, network certificate recognition devices, super SIM card reading devices, adjustable lighting, projection keyboards, smart temperature detectors, and biometric recognition devices: including facial recognition, iris recognition, voiceprint recognition, etc. Business handling information refers to information used to indicate the start of using the smart display device to handle business. User identity information refers to the identity information of the user in the user management system connected to the smart display device.

[0027] Specifically, during the use of the smart display device, when a device connected to the smart display device is called or a service program of the smart display device is run, information indicating the start of using the smart display device to handle business, i.e., business handling information, is generated. In response to the business handling information, a device for identifying the user's identity connected to the smart display device is called to run to identify the identity of the current user. The device for identifying the user's identity may be a high-definition camera, a monocular / binocular portrait camera, and a biometric recognition device. Through the recognition of the device for identifying the user's identity, the user identity information is obtained, i.e., the user's identity information in the user management system connected to the smart display device. For example, if the identity information in the user management system is the user's name, then the corresponding user identity information is also the user's name.

[0028] Furthermore, if the business handling information indicates that the device connected to the called smart display device is a multi-document reading device, a card reading and writing device, a fingerprint recognition device, a network certificate recognition device or a super SIM card reading device, the user identity information is obtained from the customer management system through the device connected to the currently called smart display device. In addition, not all devices connected to the called smart display device or all service programs running the smart display device will generate business handling information. Business handling information will only be generated when the called device and the running service program are devices or programs used to serve customers, rather than work devices or work programs of staff.

[0029] S20: Obtain available device information based on user identity information and business handling information.

[0030] In this embodiment, the available device information refers to the currently available devices for collecting user behavior information.

[0031] Specifically, the business processing information indicates calling a device connected to the smart display device or running a service program of the smart display device. Therefore, based on the business processing information, continue to judge the detailed business that the current user needs to handle. For example, by monitoring the instructions for calling the device or the specific functions of the running service program, the detailed business information that the current user needs to handle is obtained. Based on the detailed business information, the business processing device corresponding to the detailed business information is obtained. The business processing device corresponding to the detailed business information is not used to collect user behavior. The currently available device for collecting user behavior information is judged. For example, when a user handles an account query service, the user is required to enter the account information. At this time, the device related to the business handled by the user There is a document reading device, therefore, the currently available devices for collecting user behavior information include high-speed scanners and monocular / binocular portrait cameras; in addition, according to the user identity information, the corresponding device information for collecting user behavior information that the user dislikes is obtained from the preset user dislike device table. For example, the user feels that the monocular / binocular portrait camera is parallel to the user's eyes, so the user has a resistance to the collection of the monocular / binocular portrait camera. Therefore, it is determined whether the currently available device for collecting user behavior information is different from the device for collecting user behavior information that the user dislikes, and the different device is used as the currently available device for collecting user behavior information, that is, the available device information. In this embodiment, the machine learning model analyzes and learns the user's emotions through the user's historical behavior when handling different businesses, and derives the information of the device for collecting user behavior that is being used when it is determined that the user feels disgusted, resistant or disgusted, etc., to form a user dislike device table representing the devices for collecting user behavior that each user is prohibited from using when handling each business.

[0032] Furthermore, among the devices that collect user behavior, the image and video acquisition devices can collect more user behavior information. Therefore, if the detailed business information that the current user needs to handle requires the use of all image and video acquisition devices of the smart display device, at this time, the image and video acquisition devices are not used to collect user behavior, which will result in a small amount of user behavior information obtained; therefore, all image and video acquisition devices are regarded as currently available devices for collecting user behavior information, and then screened by user identity information to obtain available device information.

[0033] S30: Collect user behavior information based on available device information.

[0034] In this embodiment, user behavior information refers to the action information of the user in the process of handling business.

[0035] Specifically, based on the available device information, the corresponding device is controlled to collect the action behavior information of the user in the process of handling the business, so as to obtain the user behavior information.

[0036] In this embodiment, before the smart display device is put into use, each device for collecting user behavior information connected to the smart display device is tested to determine the specific information that can be collected by each device for collecting user behavior information. According to local user privacy protection regulations, the working parameters of each device for collecting user behavior information are adjusted or devices that violate local user privacy protection regulations are prohibited. For example, the angle of the monocular / binocular portrait shooting camera is adjusted so that the image captured by the monocular / binocular portrait shooting camera does not contain privacy information such as the password entered by the user.

[0037] S40: Input the user identity information and user behavior information into a machine learning model to obtain user habit information and device control information. The user device control is used to optimize the interaction mode between different devices and users according to user habits.

[0038] In this embodiment, user habit information refers to the behavior habit information of the user in the process of handling services. Device control information refers to the information for controlling the interaction mode between the device and the user.

[0039] Specifically, the user identity information and user behavior information are input into a machine learning model. The machine learning model is a deep learning model that has the function of identifying user behavior, understanding the meaning of user behavior, and learning user habits through multiple user behavior information. Therefore, based on the machine learning model, the user's behavioral habit information when handling the current business, that is, the user habit information, and information corresponding to the user habit information for optimizing the interaction method between the device and the user, that is, the device control information, are obtained. For example, the user habit information indicates that the user is accustomed to using the page magnification function of the electromagnetic signature screen during the business handling process. The corresponding device control information is the magnification multiple of the user's historical page. When the user opens the corresponding page, the electromagnetic signature screen is controlled to automatically zoom in, or when the user stops signing during the user's use of the electromagnetic signature screen, it is automatically zoomed in when it is predicted that the user is about to use the page magnification function.

[0040] Furthermore, a user may have different behavioral habit information when handling the same business. For example, the behavioral habit information of a user when handling the same business at different time points is different. Therefore, if a user has different behavioral habit information when handling the same business, the user's current action behavior is analyzed based on the machine learning model, and the corresponding behavioral habit information is selected as the current behavioral habit information.

[0041] S50: Control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

[0042] In this embodiment, the user reaction behavior information refers to the behavior habit information when the user changes the device interaction mode during the service handling process.

[0043] Specifically, based on the device control information, the corresponding device is controlled to adjust the user's interaction mode, and the user behavior information collected after the device interaction mode is changed based on the device control information is used as the user reaction behavior information, that is, the user's behavioral habit information when the device interaction mode changes during the business process, and the user reaction behavior information is input into the machine learning model. Based on the user behavior information and the user reaction behavior information, the differences between the user behavior information and the user reaction behavior information are analyzed through the machine learning model to judge the user's satisfaction with the change in the device interaction mode. In this way, according to the user's satisfaction with the change in the device interaction mode, the device control information corresponding to the current business type is adjusted. In addition, the machine learning model is trained to learn user behavior habits based on the user behavior information and the user reaction behavior information, so as to improve the accuracy of judging user behavior habit information.

[0044] In one embodiment, if Figure 2As shown, in step S10, based on the business handling information, the corresponding device is called to identify the user identity and obtain the user identity information, which specifically includes: S11: Based on the business handling information, call the corresponding device to identify the user's identity and determine whether the user enters identity authentication information.

[0045] In this embodiment, the identity verification information refers to information input by the user indicating the user's identity.

[0046] Specifically, in the application scenarios of smart display devices, some services may not require users to enter information to indicate their identity, namely, identity verification information. For example, staff may query the business hall for users about the services that can be handled or only use the staff's authority to query information. Some services require the user to handle them in person, while some services do not require the user to handle them in person. For services that require the user to handle them in person but do not require the user to enter identity verification information, the staff generally manually verifies the user's identity. Therefore, in order to ensure the standardization of business processes and further ensure the accuracy of identifying user identities, when calling the corresponding device to identify the user's identity based on the business handling information, it is determined whether the user has entered the identity verification information.

[0047] S12: If the judgment is yes, the identity verification information is compared with the user identity recognized by the device, and according to the comparison result, an identity verification inconsistency instruction is triggered or the user identity information is obtained.

[0048] Specifically, if it is determined that the user has entered identity authentication information, it means that the current business is a type of business that needs to be handled by the user in person. Therefore, the identity authentication information is compared with the user identity recognized by the device. For example, the user image taken by the camera is compared with the user image of the identity authentication information. According to the comparison result, if the comparison result indicates that the identity authentication information is inconsistent with the user identity recognized by the device, it means that there is a risk in the current business handling. Therefore, information is sent to the staff to remind the staff that the current identity authentication information is inconsistent with the user identity recognized by the device, thereby triggering an identity verification inconsistency instruction; if the comparison result indicates that the identity authentication information is consistent with the user identity recognized by the device, it means that the current user identity is correct, and the identity authentication information is used as the user identity information.

[0049] S13: If the judgment is no, obtaining user identity information according to the user identity recognized by the device.

[0050] Specifically, if it is determined that the user has not input identity authentication information, it means that the current service is a service type that does not require the user to handle it in person, so the user identity recognized by the device is used as the user identity information.

[0051] In one embodiment, if Figure 3As shown, in step S20, available device information is obtained according to user identity information and business handling information, specifically including: S21: Obtain device locking information according to the business handling information.

[0052] In this embodiment, the device locking information refers to the device that will be called by the service currently handled by the user.

[0053] Specifically, during the business processing of the smart display device, in order to ensure the smooth progress of the user's business processing, the device that will be called by the user's current business processing is not used to collect user behavior information. Instead, user behavior information is collected through other idle devices. Therefore, according to the specific business type handled by the user indicated by the business processing information, the device that will be called when handling the business of this type is obtained, that is, the device lock information. The device lock information refers to the prohibition of the use of devices used to collect user behavior information.

[0054] S22: Acquire corresponding user device locking information according to the user identity information.

[0055] In this embodiment, the user device locking information refers to the device that is not used to collect user behavior information based on the user's behavior habits.

[0056] Specifically, according to the user identity represented by the user identity information, the corresponding device information for collecting user behavior information that the user dislikes is obtained from the preset user dislike device table, and the device that is not used to collect user behavior information based on the user's behavior habits is obtained, that is, the user device lock information.

[0057] S23: Based on the device locking information and the user device locking information, obtain available device information.

[0058] Specifically, both the device lock information and the user device lock information are devices that are not used to collect user behavior information. Therefore, in combination with the device lock information and the user device lock information, the remaining devices that can collect user behavior information are used as available device information for collecting user behavior information.

[0059] In one embodiment, if Figure 4 As shown, in step S30, based on the available device information, user behavior information is collected, specifically including: S31: Acquire device behavior control information based on available device information and user identity information.

[0060] In this embodiment, the device behavior control information refers to the control information of the working parameters of the device in the process of collecting user behavior information.

[0061] Specifically, the equipment used to collect user behavior information includes a high-speed scanner and a monocular / binocular portrait shooting camera. If the current environment of the smart display device is poor, for example, the ambient brightness is low or the background sound is noisy, then the amount of practical information in the user behavior information collected by the high-speed scanner and the monocular / binocular portrait shooting camera will be reduced. Therefore, firstly, a trial is conducted to collect user behavior information based on the available device information to obtain the trial user behavior information, analyze the trial user behavior information, extract information of the same type as the user identity information (for example, a user portrait photo) from the trial user behavior information, and compare the information of the same type as the user identity information in the trial user behavior information with the user identity information to determine whether the user's identity can be clearly determined based on the trial user behavior information. If the judgment is yes, there is no need to adjust the current equipment for collecting user behavior information. If If the judgment is no, then based on the comparison result of the same type of information as the user identity information in the test user behavior information with the user identity information, obtain the control information on the working parameters of the device in the process of collecting the user behavior information, that is, the device behavior control information. For example, the information collecting the user behavior is a camera, and the test user behavior information is a video. If the comparison result indicates that it is difficult to determine the identity of the user through the image in the video, then analyze the comparison result to determine the reason why it is difficult to determine the identity of the user, for example, if the ambient brightness is dim, then generate corresponding shooting parameters such as adjusting the exposure of the camera according to the comparison result, or control the adjustable lighting for fill light. If the information collecting the user behavior is a camera, and the test user behavior information is a video, if the comparison result indicates that it is difficult to determine the user's sound waves through the audio of the video, then generate device behavior control information for controlling the noise reduction recording device to record.

[0062] S32: Control the corresponding device based on the device behavior control information and collect user behavior information.

[0063] Specifically, the operation of the corresponding device for collecting user behavior information is controlled based on the device behavior control information, and the operation parameters of the device for collecting user behavior information are adjusted based on the device behavior control information to collect the user behavior information.

[0064] In one embodiment, if Figure 5 As shown, in step S40, the user behavior information is input into the machine learning model to obtain user habit information and device control information, specifically including: S41: Input the user identity information and user behavior information into the machine learning model, extract the corresponding user habit information and user expectation ranking information, and the user expectation ranking information is used to represent the user's demand for the business handling process.

[0065] In this embodiment, the expected sorting information refers to the user's demand information for the current business handling process.

[0066] Specifically, the user identity information and user behavior information are input into the machine learning model to analyze the user's current action behavior, extract the user habit information corresponding to the current analysis of the user's current action behavior, and extract the information representing the user's expectation level for different needs of the current business processing, that is, user expectation ranking information. For example, the user hopes that the speed of business processing will be increased or the user hopes that the waiting time during the business processing will be reduced, and the user's expectation level for increasing the speed of business processing is higher than the user's expectation level for reducing the waiting time during the business processing.

[0067] Furthermore, the machine learning models corresponding to different user behavior information are different, that is, different user behavior information collected by different devices are input into corresponding machine learning models, and the machine learning model corresponding to each user behavior information is a pre-set machine learning model that can better identify user behavior and understand the meaning of user behavior.

[0068] S42: Acquire device control information according to the user habit information and the user expectation sorting information.

[0069] Specifically, based on the user habit information and the user expectation sorting information, the corresponding device control information that conforms to the user's habits and meets the user's expectations is comprehensively judged. For example, if the user hopes that the speed of business processing will be increased or the user hopes that the waiting time in the business processing process will be reduced, then the device control information corresponding to the increased speed of business processing includes automatic information filling, step guidance and prompts, and omitting steps that are historically determined to have been completed by the user, etc. The device control information corresponding to the reduced waiting time in the business processing process also includes downloading business data in advance, calling more computing resources to process the current business, and increasing the data channel between the smart display device and the customer management system, etc. The user's expectation of increasing the speed of business processing is higher than the user's expectation of reducing the waiting time in the business processing process. Therefore, based on the user habit information, more automatically filled information, more step guidance information, and more steps that are historically determined to have been completed by the user are judged, and the amount of business data downloaded in advance is correspondingly reduced, the computing resources for processing the current business are correspondingly reduced, and the data channel between the smart display device and the customer management system is reduced.

[0070] In one embodiment, if Figure 6 As shown, in step S42, according to the user's desired sorting information, the device control information is obtained, which specifically includes: S421: The user's expected sorting information sorts the business processing speed, business waiting time, business processing flow visibility and business processing result display visibility. The business processing speed, business waiting time, business processing flow visibility and business processing result display visibility all correspond to multiple device control information with different optimization levels. The user's expected sorting information is used to control the optimization level of different devices.

[0071] In this embodiment, the service waiting time refers to the time the user waits during the service processing. The service processing process visibility refers to the visibility of the specific next step instructions when the user processes the service. The service processing result display visibility refers to the visibility of the form of displaying the service processing result.

[0072] Specifically, the user expected sorting information corresponding to each user is information for sorting the business processing speed, business waiting time, business processing process visibility and business processing result display visibility. The business processing speed, business waiting time, business processing process visibility and business processing result display visibility all correspond to multiple device control information with different optimization levels. For example, when the business processing speed ranks first, the corresponding device control information has the highest optimization level, when the business processing speed ranks second, the corresponding device control information has the second highest optimization level, and when the business processing speed ranks fourth, the corresponding device control information has the lowest optimization level. Therefore, the sorting information represented in the user expected sorting information is used to control the optimization level of different devices. The business processing process visibility refers to the visibility of the specific next operation guidance when the user handles the business, for example, the size and number of the animation demonstration guiding the next operation. The business processing result display visibility refers to the visibility of the form of displaying the business processing result, for example, the form of displaying the business processing result is graphics or text, as well as the size of the graphics and the size of the text.

[0073] In this embodiment, the type of device control information is fixed, that is, when the device control information is obtained according to the user habit information and the user expected sorting information through the machine learning model, the specific method is to associate the existing device control information with the user. Therefore, if the user habit information of different users is the same, the corresponding device control information is also the same. If the user habit information of different users is relatively similar, the corresponding device control information may also be the same.

[0074] S422: Acquire device control information of a corresponding optimization degree according to the user's desired sorting information.

[0075] Specifically, according to the order indicated by the user's desired sorting information, the device control information of the corresponding optimization degree is obtained.

[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0077] In one embodiment, a digital twin display device of a smart sign is provided, and the digital twin display device of the smart sign corresponds one-to-one to the digital twin display method of the smart sign in the above embodiment. Figure 7 As shown, the digital twin display device of the smart sign includes a user identity recognition module, an available device acquisition module, a user behavior acquisition module, a model output module and a model training module. The detailed description of each functional module is as follows: The user identity recognition module is used to call the corresponding device to identify the user's identity and obtain the user's identity information based on the business handling information; An available device acquisition module is used to obtain available device information based on user identity information and business handling information; A user behavior collection module, used to collect user behavior information based on available device information; The model output module is used to input user identity information and user behavior information into the machine learning model to obtain user habit information and device control information. The user device control is used to optimize the interaction between different devices and users according to user habits. The model training module is used to control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

[0078] Optionally, the user identification module includes: The user input judgment submodule is used to call the corresponding device to identify the user's identity based on the business handling information and determine whether the user enters the identity authentication information; The identity comparison submodule is used to compare the identity verification information with the user identity recognized by the device if the judgment is yes, and trigger the identity verification inconsistency instruction or obtain the user identity information according to the comparison result; The user identity acquisition submodule is used to obtain user identity information according to the user identity recognized by the device if the judgment is no.

[0079] Optionally, available device acquisition modules include: The first locking submodule is used to obtain device locking information according to business handling information; The second locking submodule is used to obtain corresponding user device locking information according to the user identity information; The available device acquisition submodule is used to acquire available device information based on the device locking information and the user device locking information.

[0080] Optionally, the user behavior collection module includes: A behavior control submodule, used to obtain device behavior control information based on available device information and user identity information; The collection submodule is used to control the corresponding device based on the device behavior control information and collect user behavior information.

[0081] Optionally, the model output module includes: The information extraction submodule is used to input user identity information and user behavior information into the machine learning model to extract the corresponding user habit information and user expectation ranking information. The user expectation ranking information is used to express the user's demand for the business handling process; The model output submodule is used to obtain device control information based on user habit information and user expectation sorting information.

[0082] Optionally, the user's expected sorting information sorts the business processing speed, business waiting time, business processing process visibility and business processing result display visibility. The business processing speed, business waiting time, business processing process visibility and business processing result display visibility all correspond to multiple equipment control information with different optimization levels. The user's expected sorting information is used to control the optimization level of different equipment. The model output submodule includes: The model output unit is used to obtain device control information of the corresponding optimization degree according to the information sorted by the user's expectations.

[0083] For the specific definition of the digital twin display device of the smart sign, please refer to the definition of the digital twin display method of the smart sign above, which will not be repeated here. Each module in the digital twin display device of the above-mentioned smart sign can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0084] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store user identity information, available device information, user behavior information, user reaction behavior information, user habit information and device control information, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a digital twin display method of a smart logo is realized.

[0085] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Based on the business handling information, the corresponding device is called to identify the user and obtain the user identity information; Obtain available device information based on user identity information and business handling information; Collect user behavior information based on available device information; Input user identity information and user behavior information into the machine learning model to obtain user habit information and device control information. User device control is used to optimize the interaction between different devices and users based on user habits. Control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

[0086] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Based on the business handling information, the corresponding device is called to identify the user and obtain the user identity information; Obtain available device information based on user identity information and business handling information; Collect user behavior information based on available device information; Input user identity information and user behavior information into the machine learning model to obtain user habit information and device control information. User device control is used to optimize the interaction between different devices and users based on user habits. Control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

[0087] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0088] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0089] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A digital twin display method for smart signs, characterized in that: The digital twin display method of the smart sign comprises the steps of: Acquire user location and posture information, and based on the user location and posture information, call a corresponding device to identify the user and obtain user identity information; Acquire available device information according to the user identity information and the user location and posture information; Based on the available device information, collect user behavior information; Inputting the user identity information and the user behavior information into a machine learning model to obtain user habit information and device control information, wherein the user device control is used to optimize the interaction mode between different devices and users according to user habits; Based on the device control information, the corresponding device is controlled, user reaction behavior information is collected, the user reaction behavior information is input into the machine learning model, and the machine learning model is trained through the user behavior information and the user reaction behavior information.

2. The digital twin display method of the smart sign according to claim 1 is characterized in that: The calling of the corresponding device to identify the user identity based on the business handling information to obtain the user identity information specifically includes: Based on the business handling information, the corresponding device is called to identify the user's identity and determine whether the user enters the identity authentication information; If the judgment is yes, the identity verification information is compared with the user identity recognized by the device, and according to the comparison result, an identity verification inconsistency instruction is triggered or the user identity information is obtained; If the judgment is no, the user identity information is obtained according to the user identity recognized by the device.

3. The digital twin display method of the smart signage according to claim 1 is characterized in that: The obtaining of available device information according to the user identity information and the business handling information specifically includes: According to the business handling information, obtain device locking information; According to the user identity information, obtain corresponding user device locking information; Based on the device locking information and the user device locking information, available device information is acquired.

4. The digital twin display method of the smart sign according to claim 1 is characterized in that: The collecting of user behavior information based on the available device information specifically includes: Based on the available device information and the user identity information, acquiring device behavior control information; Based on the device behavior control information, the corresponding device is controlled to collect user behavior information.

5. The digital twin display method of the smart signage according to claim 1 is characterized in that: The inputting the user identity information and the user behavior information into a machine learning model to obtain user habit information and device control information specifically includes: Input the user identity information and the user behavior information into a machine learning model to extract corresponding user habit information and user expectation ranking information, where the user expectation ranking information is used to represent the user's demand for the business handling process; Device control information is acquired according to the user habit information and the user desired sorting information.

6. The digital twin display method of the smart sign according to claim 1 is characterized in that: The acquiring device control information according to the user expected sorting information specifically includes: The user expected sorting information sorts the service processing speed, service waiting time, service processing flow visibility and service processing result display visibility. The service processing speed, service waiting time, service processing flow visibility and service processing result display visibility all correspond to a plurality of equipment control information with different optimization levels. The user expected sorting information is used to control the optimization levels of different equipment. According to the user's expected sorting information, device control information of a corresponding optimization degree is obtained.

7. A digital twin display device of a smart sign, characterized in that: The digital twin display device of the smart sign includes: The user identity recognition module is used to call the corresponding device to identify the user's identity and obtain the user's identity information based on the business handling information; An available device acquisition module, used to acquire available device information according to the user identity information and the service handling information; A user behavior collection module, used to collect user behavior information based on the available device information; A model output module, used to input the user identity information and the user behavior information into a machine learning model to obtain user habit information and device control information, wherein the user device control is used to optimize the interaction mode between different devices and users according to user habits; A model training module is used to control the corresponding device based on the device control information, collect user reaction behavior information, input the user reaction behavior information into the machine learning model, and train the machine learning model through the user behavior information and the user reaction behavior information.

8. The digital twin display device of the smart sign according to claim 7, characterized in that: The user identification module comprises: The user input judgment submodule is used to call the corresponding device to identify the user's identity based on the business handling information and determine whether the user enters the identity authentication information; The identity comparison submodule is used to compare the identity verification information with the user identity recognized by the device if the judgment is yes, and trigger an identity verification inconsistency instruction or obtain the user identity information according to the comparison result; The user identity acquisition submodule is used to obtain user identity information according to the user identity recognized by the device if the judgment is no.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the digital twin display method of a smart logo as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a digital twin display method of a smart sign as described in any one of claims 1 to 6 are implemented.

Images