Interactive prompt method, device and equipment
By constructing a three-dimensional venue model and positioning the body posture direction of the target personnel, and automatically selecting interactive personnel in the performance scene, the problem of low manual selection efficiency in the existing technology is solved, the interactive experience and accuracy are improved, and the interaction cost is reduced.
Patent Information
- Application Number
- CN202111515467.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-12-13
AI Technical Summary
The existing interactive methods of performance scenes rely on manual selection, and are inefficient and unable to accurately select the audience members participating in the interaction, resulting in poor interaction experience and high interaction costs.
By obtaining the space information of the venue and the body posture images of the target personnel, a three-dimensional place model is constructed, the body posture direction of the target personnel is positioned, and the audience members participating in the interaction are automatically selected.
It improves the interactive experience of the performance scene, reduces the interaction cost of user choices, and achieves more accurate and intelligent interactive personnel selection.
Smart Images

Figure CN114397959B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to an interactive prompting method, device and equipment. Background Art
[0002] At the performance site, performers and audiences usually interact with each other to enhance the performance atmosphere. At present, the interaction at the performance site is usually performed by performers or staff manually selecting audiences at the performance site as interactive persons for interaction. Summary of the invention
[0003] The embodiments of this specification provide an interactive prompting method, device and equipment, which can greatly improve the interactive experience of live interactions such as performances.
[0004] The implementation mode of this specification provides an interactive prompt method, which is applied to the service equipment of an interactive prompt system; the interactive prompt system also includes a collection device deployed in a designated place, and the method includes: obtaining a place image that at least includes the place spatial information of the designated place and the body posture of a target person; using the place image collected by at least two collection devices to construct a three-dimensional place model including the body posture of the target person; based on the three-dimensional place model, locating the direction pointed by the body posture of the target person in the designated place, so as to prompt users corresponding to the direction in the designated place to participate in the interaction.
[0005] The embodiment of this specification provides an interactive prompt device, which is applied to the service equipment of the interactive prompt system; the interactive prompt system also includes a collection device deployed in a designated place, and the device includes: an image acquisition module, which is used to acquire a place image that at least includes the place spatial information of the designated place and the body posture of the target person; a model construction module, which is used to use the place images collected by at least two collection devices to construct a three-dimensional place model including the body posture of the target person; a direction positioning module, which is used to locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, so as to prompt users corresponding to the direction in the designated place to participate in the interaction.
[0006] An embodiment of the present specification provides an interactive prompt system, which includes a cloud service device, and a local service device and a collection device deployed in a designated place: the local service device is used to obtain a place image that at least includes the place spatial information of the designated place and the body posture of a target person, and send the place image to the cloud service device; the cloud service device is used to use the place images collected by at least two collection devices to construct a three-dimensional place model including the body posture of the target person; and locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, so as to prompt users corresponding to the direction in the designated place to participate in the interaction.
[0007] The embodiments of this specification provide a service device, which includes at least one processor and a memory storing computer-executable instructions. When the processor executes the instructions, the steps of the method described in any one or more embodiments are implemented.
[0008] The implementation manner of this specification provides an interactive prompt method, which can construct a three-dimensional place model including the body posture of the target person by at least using the place space information including the designated place and the place image including the body posture of the target person. Correspondingly, the constructed three-dimensional place model includes the three-dimensional spatial characteristics of the body posture of the target person in the designated place. Then, the three-dimensional place model can be used to locate the direction pointed by the body posture of the target person in the three-dimensional space corresponding to the designated place, and then the user corresponding to the direction in the designated place can be accurately selected to participate in the interaction. Using the methods provided in the various embodiments of this specification, users can be selected from the scene to participate in the interaction more accurately and intelligently, making the on-site interaction more intelligent, reducing the interaction cost of interactive user selection, and improving the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The included drawings are used to provide a further understanding of the embodiments of the present specification, which constitute a part of the specification, are used to illustrate the embodiments of the present specification, and together with the text description, explain the principles of the present specification. Obviously, the drawings described below are only some embodiments of the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0010] Figure 1 A schematic diagram of interactive prompts provided for the implementation of this specification;
[0011] Figure 2 A schematic diagram of the seating area positioning for interactive prompts provided in the implementation manner of this specification;
[0012] Figure 3A flowchart of an interactive prompting method provided in an embodiment of this specification;
[0013] Figure 4 A schematic diagram of the module structure of the interactive prompting device provided in the embodiment of this specification;
[0014] Figure 5 A schematic diagram of the module structure of a service device provided in an embodiment of this specification. DETAILED DESCRIPTION
[0015] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the implementation modes of this specification will be clearly and completely described below in conjunction with the drawings in the implementation modes of this specification. Obviously, the described implementation modes are only part of the implementation modes of this specification, not all of the implementation modes. Based on the implementation modes in this specification, all other implementation modes obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.
[0016] For ease of understanding, in a scenario example provided in this specification, the scheme provided by the embodiment of this specification is described as follows with the performance venue as the designated venue. The interactive prompt method of this scenario example can be applied to an interactive prompt system. The interactive prompt system may include at least a service device and a capture device deployed in the performance venue for image capture. The service device may be a device that performs data processing, such as a server, a business terminal, etc. The capture device may be a video capture device, an image capture device, etc. The performance venue may be a concert venue, an annual meeting venue, etc. Correspondingly, the target person may be a performer, a host, a speaker, etc.
[0017] In this scenario example, Figure 1 As shown, the service equipment may include cloud service equipment and local service equipment deployed in the performance venue. Figure 2 As shown, the performance venue may include a stage area for performers and a viewing area for audiences. The viewing area may be provided with a plurality of seats and divided into a plurality of sub-seat areas in a specified direction. The specified direction may be a direction perpendicular to the viewing area toward the stage area. Figure 2 As shown, the direction from the viewing area to the stage area is the Y direction, and accordingly, the seats in the viewing area can be divided along the X direction perpendicular to the Y direction to obtain multiple sub-seat areas A, B, C, ... The cloud service device can store the area position information occupied by each sub-seat area and the area identifiers A, B, C, ..., and can also store the seat identifiers 11, 12, ... MN contained in each sub-seat area.
[0018] The cloud service device can provide ticket purchase data processing. Audience members can purchase tickets through the ticket application on the terminal device. For example, the ticket application interface of the terminal device can show the audience the seat distribution of the viewing area, and the audience can choose a seat, and after selecting a seat, submit an order to complete the ticket purchase. After the ticket purchase is completed, the cloud service device can record the audience's ticket purchase information, such as associating and storing the seat identifier purchased by the audience with the user identifier corresponding to the audience. Accordingly, the cloud service device can store the association relationship between the seat identifier of the sold seat and the user identifier.
[0019] like Figure 1 As shown in FIG, local service equipment, cameras, performance indicator lights, etc. can be deployed in the performance venue. Figure 2 As shown, multiple cameras can be deployed at least for the stage area in the performance venue to collect more comprehensive information about the stage area of the performance venue and the performers in the stage area. The camera can establish a communication connection with the local service device to transmit the collected information to the local service device. Wireless communication, such as Bluetooth, WIFI, etc., can be used to establish a communication connection between the camera and the local service device. The local service device can also establish a communication connection with the cloud service device to transmit the information collected by the camera or the data processed by the collected information to the cloud service device. The local service device and the cloud service device can establish a communication connection by remote communication, such as establishing a remote communication connection through a communication base station, a communication satellite, etc.
[0020] The camera can capture images of the stage area and send the captured images to the local server. The local service device can at least send the images captured by multiple acquisition devices for image capture of the stage area, the device parameters of each acquisition device, and the location information to the cloud service device. The cloud service device can construct a three-dimensional space model of the stage area based on the texture map and depth map collected by multiple acquisition devices, the device parameters of each acquisition device, and the location information. In order to make the three-dimensional space model more comprehensively display the three-dimensional body posture characteristics of the performers relative to the stage area, the three-dimensional space model of the stage area can be constructed as much as possible using the acquisition devices arranged for multiple acquisition perspectives. Of course, the local service device can also construct a three-dimensional space model of the stage area based on the texture map and depth map collected by each acquisition device, the device parameters of each acquisition device, and the location information, and send the constructed three-dimensional space model to the cloud service device.
[0021] At the performance site, when the performers want to interact with the audience, the performers can, for example, stretch their arms toward a certain sub-seat area to select the audience located in that sub-seat area to participate in the interaction. For example, the performers can first issue an interactive start instruction by voice, such as "Next, we will select a friend on site to come to the stage to play an interactive game together", and then the performers can render the atmosphere of the interactive user's selection by moving their arms. After selecting the sub-seat area to participate in the interaction, they can indicate that they have been selected by means of specific gestures. Accordingly, after detecting the interactive start instruction, the local service device can receive the images collected by each acquisition device of the stage area after the interactive start instruction is issued, and then perform gesture recognition on each received image. In the case of detecting a specific gesture, each image at the time when the specific gesture is issued can be extracted as the target place image, so as to use the target place image to construct a three-dimensional space model corresponding to the stage area.
[0022] The cloud service device can extract the extension direction of the performer's arm in the stage area from the three-dimensional space model of the stage area. The relative positions of the stage area and each sub-seat area of the viewing area in the performance venue are already determined. Accordingly, based on the relative positions of the stage area and each sub-seat area of the viewing area in the performance venue, the sub-seat area of the viewing area corresponding to the extension direction can be determined, and then the audience can be selected from the sub-seat area to participate in the interaction.
[0023] Alternatively, the cloud service device may pre-store a three-dimensional venue model corresponding to the performance venue. When sending the above information to the cloud service device, the local service device may send the performance venue identifier corresponding to the performance venue to the cloud service device, so that the cloud service device retrieves the three-dimensional venue model corresponding to the corresponding performance venue identifier. Alternatively, the cloud service device may also locate the performance venue where the local service device is located by analyzing the address information of the local service device, and then retrieve the three-dimensional venue model corresponding to the performance venue. The cloud service device may further replace the corresponding part in the three-dimensional venue model with the three-dimensional space model of the stage area to obtain an interactive three-dimensional venue model containing the body postures of the performers. The cloud service device may extract the extension direction of the performer's arm in the performance venue from the interactive three-dimensional venue model. The position information of each sub-seat area in the viewing area in the performance venue is also known. Accordingly, based on the position information of each sub-seat area in the viewing area in the performance venue, the sub-seat area corresponding to the extension direction of the viewing area can be determined, and then the audience can be selected from the sub-seat area to participate in the interaction.
[0024] like Figure 2As shown, the cloud service device can extract the extension direction of the performer's arm in the performance venue from the interactive three-dimensional venue model, and further extend the extension direction to the viewing area, and locate the sub-seat area B where the extension direction falls, so as to prompt the audience in the sub-seat area B to participate in the interaction. After locating the sub-seat area where the extension direction falls, the cloud service device can randomly extract a seat identifier corresponding to a sold seat from the seat identifiers contained in the corresponding sub-seat area as the interactive seat identifier.
[0025] The cloud service device can extract the user ID corresponding to the interactive seat ID based on the association between the interactive seat ID and the user ID, and send a notification of participation in the interaction to the terminal device corresponding to the user ID, so as to prompt the audience corresponding to the user ID to participate in the interaction. Figure 1 As shown, the cloud service device may also feed back the terminal device information of the user corresponding to the interactive seat identification to the local service device, so that the local service device sends a notification of participation in the interaction to the terminal device. Alternatively, the cloud service device may also feed back the interactive seat identification to the local service device, so that the local service device controls the performance indicator light to illuminate the interactive seat corresponding to the interactive seat identification, so as to prompt the audience corresponding to the interactive seat to participate in the interaction. Alternatively, after confirming the seat information of the audience participating in the interaction, the seat identification and position information of the audience participating in the interaction may also be presented on the stage screen, so that the performers can quickly understand the position of the audience participating in the interaction.
[0026] Performers can interact with the audience participating in the interaction on the spot, such as singing together, going on stage to participate in activities, etc. Of course, the on-site service personnel can also associate the interactive activity information that needs to be participated in with the performance venue identification, and send it to the cloud service device through the ticketing application of the terminal device. When the cloud service device feeds back the interactive seat identification or the terminal device information of the user corresponding to the interactive seat identification to the local service device, it can also feed back the interactive activity information corresponding to the corresponding performance venue identification to the local service device. The local service device can control the interactive activity information to be displayed on the stage screen, or it can also send the interactive activity information to the terminal device of the audience participating in the interaction for display on the terminal device. The specific interactive implementation method can be configured according to demand and is not limited here.
[0027] By using the above implementation scheme, the audience members who participate in the interaction can be quickly selected based on the on-site posture information of the performers and the spatial information of the performance site, making the on-site interaction more intelligent, reducing the interaction cost of interactive user selection, and improving the user's interactive experience.
[0028] Of course, the service device can also be only a local service device. The cloud service device can pre-associate the seat identification purchased by the audience with the user identification and send it to the local service device so that the local service device can store it. The local service device can also store the three-dimensional venue model of the corresponding performance venue, the location information of the sub-seat area, the seat identification contained in each sub-seat area, etc. Correspondingly, the data processing of the interactive prompt is handled by the local service device. The specific implementation method can refer to the above scenario example, which will not be repeated here.
[0029] Of course, the service device can also be only a cloud service device. The acquisition device can directly establish a communication connection with the cloud service device to send the collected information directly to the cloud service device, so that the cloud service device can execute the steps of building a three-dimensional spatial model of the stage area, locating the direction pointed by the performers' body postures in the performance venue, etc., and after obtaining the interactive seat identification, send a notification of participation in the interaction to the terminal device of the audience corresponding to the interactive seat identification, etc., to prompt the relevant audience to participate in the interaction. The specific implementation method can refer to the above scenario example, which will not be repeated here.
[0030] Figure 3 A flowchart of an interactive prompting method provided by an embodiment of this specification is shown. Based on the above scenario example, Figure 3 As shown, the embodiment of this specification also provides an interactive prompting method, which is applied to the service equipment of the interactive prompting system, and the method may include the following steps.
[0031] S302: Acquire a place image that at least includes the place space information of the designated place and the body posture of the target person.
[0032] The designated place may be, for example, a performance venue, an annual meeting venue, a competition venue, etc. The designated place may be a relatively closed place or an open place. For example, the performance venue may be a stadium for holding a concert, or an outdoor area temporarily built for holding a concert. The target person may be, for example, a performer, a speaker, a host, etc. The target person may be pre-specified. For example, a person located in a designated area in a designated place may be used as the target person, such as a person located in the stage area of a performance venue may be used as the target person. Alternatively, a person with certain characteristics may be used as the target person, such as configuring the target person's facial information in the service device, and determining the target person by comparing the facial information. Of course, the method for determining the target person may also be configured according to the actual situation of the application scenario.
[0033] The acquisition device can acquire images for a designated place. In the embodiments of the present specification, the images acquired by the acquisition device for a designated place can be described as place images. The acquisition device can send the acquired images to the service device. For example, in the case where the service device includes a local service device and a cloud service device, the acquisition device can transmit the acquired images to the local service device by wireless communication. Of course, in the case where the service device only includes a cloud service device, the acquisition device can also transmit the acquired images to the cloud service device by remote communication.
[0034] The service device can at least use the place image containing the body posture of the target person and the place space information of the designated place to construct a three-dimensional space model containing the three-dimensional space information of the body posture of the target person in the designated place, so as to more accurately locate the direction pointed by the body posture of the target person in the designated place. For example, if a person located in a designated area is pre-designated as a target person, the service device can at least receive the place images collected by multiple acquisition devices for the designated area to construct a three-dimensional space model containing the three-dimensional space information of the body posture of the target person in the designated place, such as the three-dimensional space model constructed for the stage area in the above scene example. If the target person is determined based on face information, the service device can further filter out the place image containing at least the place space information of the designated place and the body posture of the target person based on face information comparison after receiving the place images sent by each acquisition device. Of course, in the application scenario where the target person is determined based on other methods, other place image screening methods can also be set accordingly, which is not limited here.
[0035] In some other embodiments, the service device can also select a preset time interval including the time when the interactive start instruction is issued when the interactive start instruction is detected; and obtain the place image including the place space information of the designated place and the body posture of the target person collected within the preset time interval. The number of images collected by the acquisition device on the scene is usually very large, and the time and number of on-site interactions are very small. If the body posture positioning of the target person is performed in real time, it will cause a huge waste of resources. In addition, the postures that performers and speakers may appear on the scene are diverse. Even if the target person stretches his arms to a sub-seat area, it does not mean that the target person wants to interact with the users in the sub-seat area at this time, which may also cause frequent interaction triggering and bring a bad experience. In this embodiment, by pre-configuring the interactive start instruction, the body posture positioning processing of the target person is performed when the interactive start instruction is detected, which can greatly reduce the waste of system resources and improve the interactive experience.
[0036] If a performer wants to interact with the audience, the performer can first say a voice message of "Let's choose a friend to sing together on the spot", and then the performer can select the sub-seat area to participate in the interaction by stretching his arm. Accordingly, the voice message sent by the performer can be used as an interaction start instruction. For example, the voice information of the performers, hosts, etc. that usually start the interaction can be collected in advance, and features can be extracted to construct an interaction start voice instruction set. Correspondingly, the voice acquisition device can collect the voice information of the performers in real time and send it to the service device. The service device can detect whether the performer has issued an interaction start recognition instruction based on the interaction start voice instruction set. After detecting that the voice information is an interaction start instruction, the service device can extract the place space information of the designated place and the body posture of the target person from the received place image to construct a three-dimensional place model.
[0037] Alternatively, the performer may first select the sub-seat area to participate in the interaction by stretching his arm, and in the process, send out a language message such as "Let's select a friend from this area to sing together, okay?" Accordingly, when the service device detects that the voice message is an interaction start instruction, it can select a preset time interval including the time when the interaction start instruction is issued, and obtain the place image including the place space information of the designated place and the body posture of the target person collected within the preset time interval to construct a three-dimensional place model.
[0038] Of course, the above processing methods are only preferred examples, and there may be other modified implementation methods during specific implementation, but as long as the functions and effects achieved are the same or similar, they should be covered within the scope of protection of this specification. For example, the interactive start instruction can also be issued by a service personnel, and the interactive start instruction can also be issued by triggering other devices, and so on.
[0039] In some other embodiments, when the service device detects an interactive selection instruction, it can also extract the place image at the time when the interactive selection instruction is issued from the received place image as the target place image; accordingly, the target place image is used to construct a three-dimensional place model including the body posture of the target person.
[0040] For example, the performer can first issue an interactive start instruction by voice, such as "Next, we will select a friend on site to come to the stage to play an interactive game together", and then the performer can render the atmosphere selected by the interactive user by moving his arms. After selecting the sub-seat area to participate in the interaction, he can indicate that he has selected it by means of specific gestures. Correspondingly, after detecting the interactive start instruction, the service device can receive the place image collected by the acquisition device after the interactive start instruction is issued, and filter out the place image containing the place space information of the designated place and the body posture of the target person. Afterwards, gesture recognition is performed based on the filtered place image. When a specific gesture is detected, the place image in which the specific gesture is detected can be used as the target place image, so as to construct a three-dimensional place model using the target place image. Among them, the implementation method of detecting the interactive selection instruction based on a specific gesture can refer to the above-mentioned implementation method of detecting the interactive start instruction based on voice information, which will not be repeated here.
[0041] Alternatively, the service device can also identify the body movement features of the target person in multiple consecutive frames of venue images, and determine whether the target person has issued an interactive selection instruction based on the body movement features of the target person. For example, when a performer wants to interact with the audience, he can stretch and move his arms. After selecting the sub-seat area to participate in the interaction, he can fix the arm extension action at this position for n seconds. Accordingly, the arm can be stretched and moved for m seconds and then fixed for n seconds as an interactive selection instruction. When the service device detects that the target person's arm stretches and moves for m seconds and then is fixed for n seconds based on the venue images at multiple consecutive acquisition times, it can be determined that the performer has issued an interactive selection instruction, that is, the interactive sub-seat area has been selected. Accordingly, the service device can select any one or more of the venue images corresponding to the n seconds as the target venue image to construct a three-dimensional venue model.
[0042] Through the above implementation, the on-site interactive atmosphere can be further rendered. At the same time, by pre-screening the target place image and constructing a three-dimensional place model based on the target place image, the image processing range of interactive posture positioning can be further clarified, thereby further improving the accuracy and efficiency of interactive posture positioning.
[0043] Of course, the above processing method is only a preferred example, and there may be other changes in the implementation. However, as long as the functions and effects achieved are the same or similar, they should be covered within the scope of protection of this specification.
[0044] S304: Using the scene images acquired by at least two acquisition devices, construct a three-dimensional scene model including the body posture of the target person.
[0045] The service device can obtain the place images collected by at least two acquisition devices, and use the place images collected by the at least two acquisition devices to construct a three-dimensional place model containing the body posture of the target person. For example, the service device can extract the body posture of the target person and the texture features and depth features of the scene around the target person from the place image containing the body posture of the target person and the place space information of the designated place, and then further combine the device parameters, location information, etc. of the acquisition device, and use the three-dimensional scene reconstruction technology to construct a three-dimensional space model containing the three-dimensional body posture features of the target person in the designated place. For the convenience of description, the constructed three-dimensional space model can be described as a three-dimensional place model. In order to make the three-dimensional body posture features of the target person in the designated place more comprehensive and accurate, the place images collected by the acquisition devices of multiple perspectives can be used as much as possible to construct a three-dimensional place model; or, other place images that do not contain the body posture of the target person can be further combined to construct a three-dimensional place model.
[0046] In the actual image acquisition process, there may be a situation where the image proportion of the target person contained in some images is relatively small, so that the body posture characteristics of the target person are not obvious, and it is difficult to accurately extract the body posture of the target person. Among them, the image proportion may refer to the proportion of the contour image of the target person in a certain image. For example, the ratio of the area of the contour image F of the target person to the area of the image E can be used as the image proportion of the target person in the image E. Of course, the image proportion can also be calculated in other ways, which are not limited here. Correspondingly, in some implementations, for the above-mentioned place images collected by the acquisition device at a certain acquisition time t, the image proportion of the target person in the place image can also be calculated, and the first K place images with a large image proportion can be screened out, and K is an integer greater than or equal to 1. Then, a three-dimensional place model can be constructed based on the first K place images screened out. The construction of a three-dimensional place model based on the place images with a relatively large image proportion of the target person can make the body posture positioning of the target person more accurate and efficient.
[0047] Of course, the above processing method is only a preferred example, and there may be other changes in the implementation. However, as long as the functions and effects achieved are the same or similar, they should be covered within the scope of protection of this specification.
[0048] S306: Locate the direction pointed by the body gesture of the target person in the designated place based on the three-dimensional place model, so as to prompt users corresponding to the direction in the designated place to participate in the interaction.
[0049] When the three-dimensional venue model is obtained, the service device can further extract the three-dimensional body posture features of the target person from the three-dimensional venue model. For example, the contour image of the target person can be located from the three-dimensional venue model first, and then the three-dimensional body posture features of the target person can be extracted from the contour image based on the key nodes of the human body. Alternatively, a human body posture detection model can be constructed using a deep learning network to extract the three-dimensional body posture features of the target person from the three-dimensional venue model using the human body posture detection model. Of course, other methods can also be used to extract the three-dimensional body posture features of the target person.
[0050] After extracting the three-dimensional body posture features of the target person, the service device can further locate the direction in which the body posture of the target person is pointing in the designated place based on the three-dimensional body posture features. For example, if the body posture of the target person is with the arm stretched out, the direction in which the arm of the target person is pointing in the designated place can be located based on the three-dimensional spatial features of the target person's arm stretching relative to the designated place. Of course, the body posture of the target person can also exist in other situations, such as when a performer points to a sub-seat area with his finger, the direction in which the performer's finger is pointing in the designated place can also be located based on the three-dimensional spatial features of the performer's finger relative to the designated place, and so on.
[0051] In some other embodiments, before locating the direction pointed by the body posture of the target person in the designated place, the service device may first identify whether the body posture of the target person is an interactive posture; if the body posture of the target person is an interactive posture, the service device may then locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model. By further determining the interactive posture, the accuracy of interactive selection positioning can be further improved.
[0052] For example, the interactive gesture is that the arm of the target person is stretched toward the area where the user is located in the designated place. Accordingly, whether the body gesture of the target person is an interactive gesture can be determined by judging whether the arm of the target person is stretched toward the area where the user is located in the designated place. Alternatively, an interactive gesture feature set can be pre-constructed, and the interactive gesture feature set can store gesture features corresponding to pre-specified interactive gestures. For example, the interactive gesture feature set can store gesture features of the target person's arm stretching toward the area where the user is located in the designated place. The extracted body gesture features can be compared with the interactive gesture feature set. If the feature matching degree of the extracted body gesture features and each gesture feature in the interactive gesture feature set is greater than the specified matching degree threshold, it can be confirmed that the body gesture of the target person is an interactive gesture. By determining the body gesture of the target person based on the interactive gesture feature set, it can be determined more quickly and accurately whether the body gesture of the target person is an interactive gesture.
[0053] When the body posture of the target person is confirmed as an interactive posture based on the three-dimensional place model corresponding to a certain acquisition time t, the body posture of the target person is confirmed as an interactive posture. The body posture of the target person can also be further judged as an interactive posture in combination with the three-dimensional place model corresponding to at least one acquisition time before and after the acquisition time t. For example, when the body posture of the target person is confirmed as an interactive posture based on the three-dimensional place models corresponding to two or more consecutive acquisition times, the body posture of the target person can be confirmed as an interactive posture to further improve the accuracy of interactive posture recognition.
[0054] After locating the direction pointed by the body gesture of the target person in the designated place, the service device can further determine the user corresponding to the direction in the designated place to prompt the user corresponding to the direction in the designated place to participate in the interaction. For example, if the interactive gesture is the target person's arm extending toward the area where the user is located in the designated place, then the area where the user corresponding to the extending direction is located in the designated place can be determined based on the position information of the area where the user is located in the designated place and the extending direction of the target person's arm, and a user is selected from the area where the user is located corresponding to the extending direction as an interactive user, and the interactive user is prompted to participate in the interaction.
[0055] As described in the above scenario example, when the designated place is a performance place, the designated area may include the stage area where the performers are located, and the viewing area where the audience are located. An interactive three-dimensional place model may be constructed based on the implementation method given in the above scenario example. The interactive three-dimensional place model contains the three-dimensional feature information of the performers' body postures in the designated place. The service device can extract the extension direction of the performer's arm in the designated place from the interactive three-dimensional place model, and then determine the seat range of the viewing area corresponding to the extension direction, and select the audience from the seat range to participate in the interaction. Figure 2 As shown, the viewing area can be divided into multiple sub-seat areas in a specified direction. The service device can extract the extension direction of the performer's arm in the specified place from the interactive three-dimensional place model, and further extend the extension direction to the viewing area, and locate the sub-seat area B where the extension direction falls, so as to prompt the audience in the sub-seat area B to participate in the interaction.
[0056] Alternatively, the designated area may only include the viewing area where the audience is located, while the area where the target person is located is relatively flexible. In this case, the location information of the target person in the designated place may be further collected using the location collection device, and the location collection device then uploads the collected location information of the target person to the service device. The service device may further locate the direction in which the target person's arm points to the viewing area in the designated place in combination with the location information of the target person and the arm extension direction located based on the above-mentioned three-dimensional place model, and then determine the seat range of the viewing area corresponding to the extension direction, and select audience members from the seat range to participate in the interaction.
[0057] Accordingly, in some embodiments, the designated place may include a viewing area where the user is located, and a plurality of seats are arranged in the viewing area. Accordingly, the service device may further extract some seats in the viewing area corresponding to the above-mentioned positioning direction to prompt the users corresponding to the some seats to participate in the interaction.
[0058] For example, the service device may store the seat identifiers contained in each sub-seat area. After locating the sub-seat area in which the extension direction falls, the service device may randomly extract one or more seat identifiers from the seat identifiers contained in the corresponding sub-seat area, use the extracted seat identifiers as interactive seat identifiers, and describe the seats corresponding to the interactive seat identifiers as interactive seats. In the case where the service device includes a local service device and a cloud service device, the cloud service device may feed back the interactive seat identifier to the local service device, and the local service device may control the performance indicator light to illuminate the interactive seat corresponding to the interactive seat identifier, so as to prompt the audience corresponding to the interactive seat to participate in the interaction. Alternatively, the local service device may also control the stage screen to display the interactive seat corresponding to the interactive seat identifier, so as to prompt the audience corresponding to the interactive seat to participate in the interaction.
[0059] Alternatively, some seats in the viewing area corresponding to the above-mentioned positioning direction may be displayed on the stage screen, and the performers may select one or more seats from the some seats corresponding to the above-mentioned positioning direction as interactive seats, and display the interactive seats on the stage screen to prompt the audience corresponding to the interactive seats to participate in the interaction.
[0060] In actual situations, there may be a situation where there are no users in some seats, so that no users can participate in the interaction in the selected interactive seats, and it is necessary to additionally select interactive seats, which will cause a waste of resources and bring a poor user experience. In some other embodiments, the service device can also store the association between seats and users. Accordingly, the service device can select at least one seat associated with the user from the said part of seats as an interactive seat, so as to prompt the user corresponding to the said interactive seat to participate in the interaction, and further reduce the interaction cost of the interactive user selection.
[0061] For example, the service device may store the association between the seat identifier corresponding to the sold seat and the user identifier that purchased the seat. The service device may randomly select one or more seat identifiers from the seat identifiers corresponding to the sold seats as interactive seat identifiers. After selecting the interactive seat identifier, the user identifier associated with the interactive seat identifier may be obtained, and a notification to participate in the interaction may be sent to the terminal device corresponding to the user identifier to prompt the user corresponding to the user identifier to participate in the interaction. Of course, the service device may also prompt the user corresponding to the interactive seat to participate in the interaction by controlling the performance indicator light to illuminate the interactive seat corresponding to the interactive seat identifier.
[0062] Alternatively, the service device may also obtain the captured image of the seating area by the capture device at the time of the interactive posture capture, and extract the seat with the user based on the captured image of the seating area, determine it as the seat associated with the user, and randomly select one or more seat identifiers from the seats associated with the user as interactive seat identifiers.
[0063] In other implementations, when there are more than two interactive seats, the users corresponding to the two or more interactive seats can be used as the interactive users who ultimately participate in the interaction. Alternatively, the users associated with the interactive seats can also be used as alternative interactive users. Performers can select one user from the alternative interactive users on site as the interactive user who ultimately participates in the interaction. Alternatively, the service device can send an interactive preemptive prompt to the terminal device of the alternative interactive user; and upon receiving the interactive selection instruction based on the interactive preemptive prompt feedback from the terminal device, based on the order in which the interactive selection instructions are received, the alternative interactive user corresponding to the first received interactive selection instruction is selected as the interactive user who ultimately participates in the interaction. In this way, the interactive cost of interactive user selection can be further reduced, and the user's interactive experience can be improved.
[0064] For example, the cloud service device can feed back the terminal device information of the interactive user corresponding to the interactive seat identification to the local service device, and the local service device sends a notification of participation in the interaction and an interactive preemptive prompt to the ticketing application in the terminal device of the candidate interactive user based on the received terminal device information. The terminal device information may refer to information for communication positioning of the terminal device. The terminal device information may include the terminal device IP address, port number, etc., so that the service device sends the notification of participation in the interaction to the ticketing application of the terminal device. Of course, the terminal device information may also be a user identifier, a terminal device identifier, etc., so that the local service device retrieves the pre-stored terminal device IP address, port number, etc. based on the user identifier, the terminal device identifier, etc., and sends the notification of participation in the interaction to the ticketing application of the corresponding terminal device. After receiving the notification of participation in the interaction and the interactive preemptive prompt, the ticketing application of the terminal device can display the notification and the interactive preemptive prompt to the candidate interactive user. For example, the ticketing application of the terminal device can display the "interactive preemptive prompt" button to the candidate interactive user. When the ticketing application detects that the candidate interactive user triggers the "interactive preemptive prompt" button, the terminal device of the candidate interactive person can send an interactive selection instruction to the local service device. The local service device may select the candidate interactive user corresponding to the interactive selection instruction received first as the interactive user who finally participates in the interaction based on the order in which the interactive selection instructions are received. The local service device may send an interactive selection notification to the terminal device of the interactive user to prompt the interactive person to participate in the interaction. Alternatively, the local service device may also control the on-site indicator light to illuminate the interactive seat corresponding to the interactive person to prompt the interactive user to participate in the interaction.
[0065] Accordingly, in some embodiments, the service device may control the on-site indicator light to illuminate the interactive seat to prompt the user corresponding to the interactive seat to participate in the interaction; or the service device may control the on-site indicator light to illuminate the interactive seat corresponding to the interactive user to enable the interactive user to participate in the interaction. Alternatively, the service device may send an interactive prompt message to the terminal device of the user associated with the interactive seat to prompt the user corresponding to the interactive seat to participate in the interaction; the service device may send an interactive prompt message to the terminal device of the interactive user to prompt the interactive user to participate in the interaction.
[0066] Of course, the above processing methods are only preferred examples, and there may be other changes in the implementation method during the specific implementation, but as long as the functions and effects achieved are the same or similar, they should be covered within the scope of protection of this specification. For example, the viewing area may not include seats, but only include multiple divided sub-viewing areas. The service device can also extract the sub-viewing area corresponding to the direction of the above positioning, and the performers can select a spectator from the sub-viewing area on site to participate in the interaction. Alternatively, the user can watch the performance online, and multiple viewing screens can also be set in the designated place, and the video connection interface of the designated multiple spectators can be displayed on each viewing screen. Correspondingly, the service device can also extract the viewing screen corresponding to the direction of the above positioning to prompt the spectators displayed on the viewing screen to participate in the interaction online. Of course, in other application scenarios, the user can also be other types of users, such as employees (the designated place is the company's annual meeting venue), etc.
[0067] Performers can interact with users who participate in the interaction on site, such as singing together, going on stage to participate in activities, etc. Of course, on-site service personnel can also associate the interactive activity information that needs to be participated in with the performance venue logo, and send it to the service device through the ticketing application of the terminal device. The service device can control the interactive activity information to be displayed on the stage screen, or it can also send the interactive activity information to the terminal devices of the audience participating in the interaction for display on the terminal devices. The specific interactive implementation method can be configured according to needs and is not limited here.
[0068] For example, the local service device may at least be a service device that has established a communication connection with the ticketing application in the user's terminal device. After determining the interactive seat identifier, the cloud service device may feed back the user's terminal device information corresponding to the interactive seat identifier to the local service device. The local service device sends a notification of participation in the interaction to the ticketing application in the terminal device of the corresponding user based on the received terminal device information. Accordingly, after receiving the notification, the ticketing application in the terminal device may display the notification to the user to prompt the user to participate in the interaction.
[0069] After receiving the notification of participating in the interaction, the ticketing application of the terminal device can also show the user a "recording" button. When the ticketing application of the terminal device detects that the user has triggered the "recording" button, the user can be prompted to establish a communication connection with the speaker device at the performance site. The user can turn on Bluetooth, WIFI, etc. in the terminal device to establish a communication connection between the terminal device and the speaker device at the performance site. Accordingly, the user's terminal device can be used as a microphone to enable the user to use the terminal device to sing with the performers. Alternatively, the user can also turn on Bluetooth, WIFI, etc. in the terminal device in advance to establish a communication connection between the terminal device and the speaker device at the performance site; accordingly, after receiving the notification of participating in the interaction, the ticketing application of the terminal device can show the user a "recording" button, and the user can trigger the "recording" button to connect his terminal device to the speaker device as a microphone. It should be noted in particular that the "record" button in the ticketing application of the terminal device is hidden before the terminal device receives a notification to participate in the interaction; after the terminal device receives the notification to participate in the interaction, it is displayed to the user, so that the user who is finally selected to participate in the interaction can participate in the chorus, while other users cannot participate in the chorus, thereby improving the user interaction experience.
[0070] Accordingly, in some embodiments, the service device may send a microphone on instruction to the terminal device of the user associated with the interactive seat, so that the corresponding terminal device displays a microphone access control. The microphone access control may be, for example, a "recording" button displayed in the ticketing application of the terminal device, or other types of controls. The terminal device may access the speaker device at the performance site based on the user's triggering of the microphone access control. The specific access method may be implemented in the above scenario example. Afterwards, the user may use his terminal device to sing with the performers. By further controlling the corresponding user's terminal device to be transformed into the user's microphone after selecting the user to participate in the interaction, the convenience of user interaction and the interactive effect may be further improved, thereby improving the user's interactive experience.
[0071] When there are more than two interactive seat identifiers, the corresponding two or more users can simultaneously serve as interactive users and sing with the performers using their respective terminal devices; of course, the performers can also designate a user from the users associated with the interactive seat identifier as an interactive user to participate in the chorus; or, the interactive user who ultimately participates in the interaction can be selected through the above-mentioned interactive preemptive method, and the service device sends a microphone-on command to the terminal device of the interactive user, so that the interactive user can use his terminal device to participate in the chorus.
[0072] The interactive prompt method provided in the above-mentioned embodiment can construct a three-dimensional place model including the body posture of the target person by at least using the place image including the place space information of the designated place and the body posture of the target person. Correspondingly, the constructed three-dimensional place model includes the three-dimensional spatial characteristics of the body posture of the target person in the designated place. Then, the three-dimensional place model can be used to locate the direction pointed by the body posture of the target person in the three-dimensional space corresponding to the designated place, and then the user corresponding to the direction in the designated place can be accurately selected to participate in the interaction. Using the methods provided in the various embodiments of this specification, users can be selected from the scene to participate in the interaction more accurately and intelligently, making the on-site interaction more intelligent, reducing the interaction cost of interactive user selection, and improving the user's interactive experience.
[0073] Based on the interactive prompting method provided in the above embodiment, the embodiment of this specification also provides an interactive prompting device applied to the service device. Figure 4 As shown, the interactive prompting device includes the following modules: an image acquisition module 402, which is used to acquire a place image that at least includes the place spatial information of the designated place and the body posture of the target person; a model construction module 404, which is used to construct a three-dimensional place model including the body posture of the target person using the place image acquired by at least two acquisition devices; a direction positioning module 406, which is used to locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, so as to prompt the user corresponding to the direction in the designated place to participate in the interaction. In the device provided in this embodiment, the functions and effects realized by the functional modules involved can be explained in comparison with other embodiments and will not be repeated here.
[0074] Based on the interactive prompt method provided in the above embodiment, the embodiment of this specification also provides an interactive prompt system, which includes a cloud service device, and a local service device and a collection device deployed in a designated place. The communication method and interaction method between the cloud service device, the local service device, and the collection device can refer to the above scenario examples and embodiments, and will not be repeated here.
[0075] In some embodiments, the local service device can be used to obtain a place image that at least includes the place space information of the designated place and the body posture of the target person, and send the place image to the cloud service device. The cloud service device can be used to construct a three-dimensional place model including the body posture of the target person using the place images collected by at least two collection devices; and locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, so as to prompt users corresponding to the direction in the designated place to participate in the interaction.
[0076] In some other embodiments, the designated place includes a viewing area where the user is located, and a plurality of seats are arranged in the viewing area. The cloud service device is also used to extract some seats in the viewing area corresponding to the direction to prompt users corresponding to the seats to participate in the interaction.
[0077] In some other embodiments, the cloud service device is further used to select at least one seat associated with a user from the portion of seats as an interactive seat, so as to prompt the user corresponding to the interactive seat to participate in the interaction.
[0078] In some other embodiments, the cloud service device is also used to feed back the seat position information of the interactive seat to the local service device. The seat position information of the interactive seat can be represented by a seat identifier, or by the sub-seat area to which the interactive seat belongs and the relative position of the interactive seat in the sub-seat area. Of course, other methods can also be used to represent it, so as to quickly locate the relative position information of the interactive seat in a designated place. The local service device is also used to receive the seat position information of the interactive seat, and control the on-site indicator light to illuminate the interactive seat based on the seat position information, so as to prompt the user corresponding to the interactive seat to participate in the interaction.
[0079] In some other embodiments, the cloud service device is further used to extract the terminal device information of the user associated with the interactive seat, and send the extracted terminal device information to the local service device. The local terminal device is further used to receive the terminal device information, and send interactive prompt information to the terminal device corresponding to the terminal device information, so as to prompt the user corresponding to the interactive seat to participate in the interaction.
[0080] In the system provided by this embodiment, the functions and effects achieved by the functional modules involved can be explained in comparison with other embodiments and will not be repeated here. The data processing performed by each device in the interactive prompt system and the interaction mode between each device can also be changed with reference to the above embodiments, but as long as the functions and effects achieved are the same or similar, they should be covered within the protection scope of this specification.
[0081] By combining cloud service equipment and local service equipment, the stability of data transmission can be guaranteed when remote communication at the performance site is not particularly good. At the same time, the cloud service equipment performs data processing such as three-dimensional space model construction, which can also reduce the requirements for the data processing capabilities of local service equipment, making the local equipment deployment simpler. It is also possible to associate interactive user choices with real-time information such as online ticket purchases, making interactive user choices more suitable for complex and changeable actual scenarios.
[0082] like Figure 5As shown, the embodiment of this specification also provides a service device, which may include at least one processor and a memory storing computer executable instructions, and the processor implements the steps of the method described in any one or more of the above embodiments when executing the instructions. The memory includes but is not limited to a random access memory (Random Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a cache (Cache), a hard disk (Hard Disk Drive, HDD) or a memory card (Memory Card). In this embodiment, the specific functions implemented by the computer program instructions in the service device can be explained in reference to other embodiments.
[0083] The above description of various embodiments of this specification is provided to those skilled in the art for the purpose of description. It is not intended to be exhaustive or to limit the present invention to a single disclosed embodiment. As mentioned above, various substitutions and variations of this specification will be apparent to those skilled in the art of the above-mentioned technology. Therefore, although some alternative embodiments have been specifically discussed, other embodiments will be apparent, or those skilled in the art will be relatively easy to derive. This specification is intended to include all substitutions, modifications, and variations of the present invention discussed herein, as well as other embodiments falling within the spirit and scope of the above-mentioned application.
[0084] This specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer service devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. Although this specification is described through implementation methods, those skilled in the art will appreciate that there are many variations and changes to this specification without departing from the spirit of this specification, and it is hoped that the appended claims will include these variations and changes without departing from the spirit of this specification.
Claims
1. An interactive prompting method, characterized in that: A service device applied to an interactive prompting system; the interactive prompting system further comprises a collection device arranged at a designated location, and the method comprises: Acquiring a place image that at least includes the place space information of the designated place and the body posture of the target person; including: in the case where an interactive start instruction is detected, selecting a preset time interval including the time when the interactive start instruction is issued, and acquiring the place image that includes the place space information of the designated place and the body posture of the target person collected within the preset time interval; or, in the case where an interactive selection instruction is detected, acquiring the place image at the time when the interactive selection instruction is issued; Using the scene images collected by at least two collection devices, construct a three-dimensional scene model including the body posture of the target person; Locating the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, selecting users in the designated place corresponding to the direction as users to participate in the interaction, so as to prompt the users to participate in the interaction; The designated place includes a viewing area where the user is located, a plurality of seats are arranged in the viewing area, and the seats in the viewing area corresponding to the direction include seats associated with the user and seats not associated with the user; the method further includes: According to the association relationship between seats and users, seats associated with users are selected from some seats corresponding to the direction in the viewing area as interactive seats, so as to prompt users corresponding to the interactive seats to participate in the interaction.
2. The method according to claim 1, characterized in that After constructing the three-dimensional place model including the body posture of the target person, the method further includes: Determining whether the body posture of the target person in the three-dimensional place model is an interactive posture; In the case that the body gesture of the target person is an interactive gesture, the direction pointed by the body gesture of the target person in the designated place is located based on the three-dimensional place model.
3. The method according to claim 2, characterized in that The interactive gesture at least includes the target person's arm extending toward the area where the user is located in the designated place.
4. The method according to claim 1, characterized in that When there are more than two interactive seats, users associated with the interactive seats are selected as candidate interactive users; the method further includes: Sending an interactive preemptive prompt to the terminal device of the candidate interactive user; When an interactive selection instruction based on the interactive preemptive prompt feedback is received from the terminal device, the candidate interactive user corresponding to the interactive selection instruction received first is selected as the interactive user based on the order in which the interactive selection instructions are received.
5. The method according to claim 1, characterized in that The method further comprises: The on-site indicator lights are controlled to illuminate the interactive seats to prompt users corresponding to the interactive seats to participate in the interaction.
6. The method according to claim 1, characterized in that The method further comprises: An interactive prompt message is sent to a terminal device of a user associated with the interactive seat to prompt the user corresponding to the interactive seat to participate in the interaction.
7. The method according to claim 1, characterized in that The method further comprises: A microphone on instruction is sent to a terminal device of a user associated with the interactive seat, so that the corresponding terminal device displays a microphone access control.
8. An interactive prompting device, characterized in that: A service device applied to an interactive prompting system; the interactive prompting system also includes a collection device arranged at a designated location, and the device includes: An image acquisition module is used to acquire a place image that at least includes the place space information of the designated place and the body posture of the target person; including: when an interactive start instruction is detected, selecting a preset time interval including the time when the interactive start instruction is issued, and acquiring the place image that includes the place space information of the designated place and the body posture of the target person collected within the preset time interval; or, when an interactive selection instruction is detected, acquiring the place image at the time when the interactive selection instruction is issued; A model building module, used to build a three-dimensional venue model including the body posture of the target person using the venue images collected by at least two collection devices; a direction positioning module, used to locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, and select users corresponding to the direction in the designated place as users to participate in the interaction, so as to prompt the users to participate in the interaction; The designated place includes a viewing area where the user is located, a plurality of seats are arranged in the viewing area, and the seats in the viewing area corresponding to the direction include seats associated with the user and seats not associated with the user; the device further includes: The screening module is used to screen out seats associated with users from some seats corresponding to the direction in the viewing area as interactive seats according to the association relationship between seats and users, so as to prompt users corresponding to the interactive seats to participate in the interaction.
9. An interactive prompting system, characterized in that: The system includes cloud service equipment, local service equipment and collection equipment deployed in designated places: The local service device is used to obtain a place image including at least the place space information of the designated place and the body posture of the target person, and send the place image to the cloud service device; The method includes: when an interactive start instruction is detected, selecting a preset time interval including the time when the interactive start instruction is issued, and obtaining a place image including the place space information of the designated place and the body posture of the target person collected within the preset time interval; or, when an interactive selection instruction is detected, obtaining a place image at the time when the interactive selection instruction is issued; The cloud service device is used to construct a three-dimensional place model including the body posture of the target person using the place images collected by at least two collection devices; and locate the direction pointed by the body posture of the target person in the designated place based on the three-dimensional place model, and select the user corresponding to the direction in the designated place as the user to participate in the interaction, so as to prompt the user to participate in the interaction; The designated place includes a viewing area where the user is located, wherein a plurality of seats are arranged in the viewing area, and the seats in the viewing area corresponding to the direction include seats associated with the user and seats not associated with the user; the cloud service device is also used to select seats associated with the user from some seats in the viewing area corresponding to the direction as interactive seats based on the association between the seats and the users, so as to prompt the user corresponding to the interactive seat to participate in the interaction.
10. The system according to claim 9, characterized in that The cloud service device is also used to feed back the seat position information of the interactive seat to the local service device; The local service device is also used to receive seat position information of the interactive seat, and control the on-site indicator light to illuminate the interactive seat based on the seat position information to prompt the user corresponding to the interactive seat to participate in the interaction.
11. The system according to claim 9, characterized in that The cloud service device is also used to extract the terminal device information of the user associated with the interactive seat, and send the extracted terminal device information to the local service device; The local terminal device is also used to receive the terminal device information and send interaction prompt information to the terminal device corresponding to the terminal device information to prompt the user corresponding to the interactive seat to participate in the interaction.
12. A service device, characterized in that: The device comprises at least one processor and a memory storing computer executable instructions, and the steps of the method according to any one of claims 1 to 7 are implemented when the processor executes the instructions.
