Projection method, apparatus, robot, readable storage medium and program product

Abstract

The application relates to a projection method, a projection device, a robot, a readable storage medium and a program product. The method comprises the following steps: in response to a projection tracking instruction for a target object, acquiring first position information of the target object, and controlling the robot to follow the target object to move based on the first position information; in the process of following the target object to move, updating a projection parameter based on second position information of the robot and a pre-established space position binding relationship, so that a display state of a projection picture on a projection carrier remains unchanged; updating modulation parameters of a multi-channel loudspeaker module based on the first position information and the updated projection parameter, so that an audio signal output by the multi-channel loudspeaker module matches the projection picture in terms of a perceived position of the target object. By using the method, the perceived position of the audio signal and the projection picture can be dynamically matched, and the audio-visual consistency of the projection picture can be ensured.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a projection method, apparatus, robot, computer-readable storage medium, and computer program product. Background Technology

[0002] Projection devices, as a common image presentation tool, are widely used in scenarios such as home entertainment, office meetings, and education and training. To achieve a complete audio-visual experience, projection devices need to output audio signals corresponding to the projected image, so that the perceived location of the sound source is spatially consistent with the sound object in the visual image.

[0003] In traditional technology, external fixed multi-channel speaker systems are typically used to achieve sound image localization in projection scenarios. This method pre-sets the spatial position of each speaker unit during the installation phase. By adjusting acoustic parameters such as gain and delay of each channel, a fixed sound field distribution is constructed at a specific listening position, allowing the listener at that position to perceive the location of the sound source that roughly matches the projected image.

[0004] However, current fixed multi-channel speaker systems rely on preset static sound field parameters for sound image localization. Once installation and parameter calibration are completed, the effective listening area of ​​the system is fixed and cannot be adaptively recalibrated during projection use. Summary of the Invention

[0005] Therefore, it is necessary to address the aforementioned technical problems by providing a projection method, device, robot, computer-readable storage medium, and computer program product capable of dynamically matching the perceived location of audio signals with the projected image, thus ensuring audiovisual consistency of the projected image.

[0006] In a first aspect, this application provides a projection method applied to a robot control module, the robot further comprising a projection module and a multi-channel speaker module, the projection module being used to project onto a projection carrier based on projection parameters to form a projected image on the projection carrier, the method comprising:

[0007] In response to a projection tracking command for a target object, the robot acquires the first position information of the target object and controls the robot to follow the target object based on the first position information.

[0008] During the movement following the target object, the projection parameters are updated based on the robot's second position information and the pre-established spatial position binding relationship, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters;

[0009] Based on the first location information and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated so that the audio signal output by the multi-channel speaker module matches the projected image with the perceived position of the target object.

[0010] In one embodiment, controlling the robot to follow the target object based on the first location information includes:

[0011] In response to the detection of multiple target objects, the robot is controlled to move to the middle area of ​​the multiple target objects based on the first position information of the multiple target objects;

[0012] Updating the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters includes:

[0013] Based on the first position information of the multiple target objects and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated so that the audio signal output by the multi-channel speaker module matches the projected image for the perceived position of the multiple target objects.

[0014] In one embodiment, the method further includes:

[0015] In response to detecting that the projected image is about to exceed the projection carrier, the projection parameters are updated so that the projected image is displayed at a smaller size on the projection carrier.

[0016] In one embodiment, updating the projection parameters based on the robot's second position information and a pre-established spatial position binding relationship includes:

[0017] Based on the first position information and / or the second position information of the robot, determine the target projection carrier;

[0018] Based on the robot’s second position information and the pre-established spatial position binding relationship, the target display position of the projected image on the target projection carrier is determined.

[0019] Based on the target display position, the projection parameters are updated to project the image onto the target display position while keeping the display state of the image unchanged.

[0020] In one embodiment, the process prior to responding to the projection tracking instruction for the target object includes:

[0021] Control the robot to move to the target location;

[0022] Control the projection module to project onto the projection carrier, and obtain the projection parameters of the projection module;

[0023] Obtain the third position information of the projection carrier, and establish a spatial coordinate system based on the third position information, the projection parameters, and the second position information of the robot;

[0024] Based on the spatial coordinate system and the preset projection adjustment strategy, the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters is determined.

[0025] In one embodiment, the robot further includes a pose awareness module, which, after controlling the robot to move to the target position, includes:

[0026] Lock the robot's posture;

[0027] Updating the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters includes:

[0028] The robot's posture change information is obtained based on the posture perception module;

[0029] Based on the posture change information, the first position information, and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated.

[0030] Secondly, this application also provides a projection device applied to a robot control module. The robot further includes a projection module and a multi-channel speaker module. The projection module is used to project onto a projection carrier based on projection parameters to form a projected image on the projection carrier. The device includes:

[0031] The robot control module is used to respond to a projection tracking command for a target object, acquire the first position information of the target object, and control the robot to follow the target object based on the first position information;

[0032] The projection parameter update module is used to update the projection parameters based on the robot's second position information and a pre-established spatial position binding relationship during the movement of following the target object, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters;

[0033] An audio parameter update module is used to update the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters, so that the audio signal output by the multi-channel speaker module matches the projected image with respect to the perceived position of the target object.

[0034] Thirdly, this application also provides a robot, including a control module, a projection module, and a multi-channel speaker module;

[0035] The projection module is used to project onto the projection carrier based on projection parameters to form a projected image on the projection carrier.

[0036] The control module is used to implement the steps of any of the above-described methods when executing a computer program.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0039] The aforementioned projection method, device, robot, computer-readable storage medium, and computer program product integrate the projection module and multi-channel speaker module into the robot. Responding to projection tracking commands, the robot is controlled to follow the target object, facilitating continuous interaction between the user and the robot. By pre-establishing a spatial positional binding relationship between the robot and the projected image, and dynamically adjusting projection parameters based on this binding relationship during the robot's movement alongside the target object, the display state of the projected image on the projection medium remains stable. Simultaneously, based on the real-time position information of the target object and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated, dynamically matching the perceived position of the audio signal output by the multi-channel speaker module with the projected image, ensuring audiovisual consistency of the projected image. Therefore, the user's viewing experience is improved. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a diagram illustrating the application environment of the projection method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating the projection method in one embodiment;

[0043] Figure 3This is a flowchart illustrating the projection method in another embodiment;

[0044] Figure 4 This is a flowchart illustrating the steps of updating projection parameters based on the robot's second position information and a pre-established spatial position binding relationship in one embodiment.

[0045] Figure 5 This is a flowchart illustrating the projection method in yet another embodiment;

[0046] Figure 6 This is a flowchart illustrating the projection method in another embodiment;

[0047] Figure 7 This is a structural block diagram of a projection device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0050] Traditional technologies typically employ external, fixed multi-channel speaker systems, requiring manual adjustment of the speaker positions relative to the projected image, which cannot adapt to scenarios with dynamically changing projection positions. Furthermore, since external projectors and speakers are independent devices, audio-visual synchronization latency is high, easily leading to lip-sync issues. Moreover, they cannot track changes in user position in real time, only being effective within a fixed "sweet spot," resulting in poor scenario flexibility.

[0051] Based on this, embodiments of this application provide a robot 00, such as... Figure 1 As shown, robot 00 includes a control module 01, a projection module 02, and a multi-channel speaker module 03. Wherein:

[0052] Projection module 02 is used to project onto a projection carrier (such as a projection wall) based on projection parameters to form a projected image on the projection carrier.

[0053] For example, the projection module 02 can be built into the body of the robot 00. The projection module 02 is electrically connected to the control module 01, and the projection module 02 is also used to output projection parameters to the control module 01 inline.

[0054] The control module 01 is used to respond to a projection tracking command for a target object (such as a user), acquire the first position information of the target object, and control the robot to follow the target object based on the first position information; during the movement of following the target object, the projection parameters are updated based on the robot's second position information and the pre-established spatial position binding relationship so that the display state of the projected image on the projection carrier remains unchanged; based on the first position information and the updated projection parameters, the modulation parameters of the multi-channel speaker module 03 are updated so that the audio signal output by the multi-channel speaker module 03 matches the perceived position of the target object with the projected image.

[0055] For example, the control module 01 is electrically connected to the multi-channel speaker module 03, which can be used to play audio signals. In one possible implementation, the control module 01 may employ a hard synchronization design, sending a modulated audio signal corresponding to the image frame to the multi-channel speaker module 03 simultaneously with sending the image frame to the projection module 02, to ensure that the audio-visual delay does not exceed 5 milliseconds.

[0056] Optionally, the robot 00 may further include at least one of a pose perception module, a vision perception module, a display module, and a sound acquisition module. Wherein:

[0057] The pose perception module is used to sense the posture changes of robot 00 and to achieve simultaneous localization and mapping (SLAM) of robot 00. Specifically, the pose perception module can be implemented based on inertial measurement units (IMUs), lidar, and ranging sensors.

[0058] The visual perception module is used to track the three-dimensional coordinates of the user's head / both ears in real time, assisting in calibrating the relative position of the robot 00 body with the projection wall and the user. Specifically, the visual perception module can be implemented based on a visual camera and a TOF (Time of Flight) depth camera.

[0059] The display module is used to display the user interface (UI) of robot 00.

[0060] The sound acquisition module is used to collect environmental acoustic parameters (reverberation time, echo intensity, or background noise, etc.).

[0061] In one exemplary embodiment, such as Figure 2 As shown, a projection method is provided. Taking the application of this method to the control module 01 mentioned above as an example, the method includes the following steps S120 to S160. Wherein:

[0062] S120, in response to the projection tracking command for the target object, acquires the first position information of the target object, and controls the robot to follow the target object based on the first position information.

[0063] The target object can be a user. The projection tracking command can be a voice command or a touch command input by the user through the robot's display module.

[0064] For example, control module 01 can respond to a projection tracking command for a target object, acquire the real-time position information of the target object collected by the robot's visual perception module, and obtain first position information; based on the first position information, determine the target position of the robot at the next moment, and control the robot to move to the target position through the robot's movement mechanism, so as to realize the robot following the target object. In one possible implementation, when the audience A moves from directly in front of the projection (azimuth angle 0°) to the right, the distance and azimuth angle data between the user and the projection screen can be dynamically updated; control module 01 simultaneously sends a command to the movement mechanism to control the robot to move slowly to the right; during the movement, control module 01 can also simultaneously scan the surrounding environment in real time through the lidar in the pose perception module to avoid obstacles.

[0065] S140, during the process of following the target object, the projection parameters are updated based on the robot's second position information and the pre-established spatial position binding relationship, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters.

[0066] The projection medium includes, but is not limited to, projection walls and projection screens. Maintaining a consistent display state can mean that the size and / or resolution remains unchanged. It should be noted that there can be one or more projection mediums in the current space. Maintaining a consistent display state on the projection medium can mean that the projected image remains unchanged while being projected onto the same position on the same medium, or it can mean that the image remains unchanged while being projected onto the nearest projection medium by the user.

[0067] For example, the robot's second position information may include the three-dimensional coordinates of the robot's center in a spatial coordinate system. When the control module 01 detects a change in the robot's second position information, it can, based on a pre-established spatial position binding relationship and with the constraint that the spatial position of the projected image remains unchanged, calculate the updated projection parameters required to maintain the unchanged display state of the projected image on the projection carrier, and send the updated projection parameters to the projection module 02 to control the projection module 02 to project according to the updated projection parameters. Here, the pre-established spatial position binding relationship can also be used to keep the distance between the user and the robot, and the distance between the robot and the wall, within a preset distance range.

[0068] S160, based on the first position information and the updated projection parameters, update the modulation parameters of the multi-channel speaker module so that the audio signal output by the multi-channel speaker module matches the perceived position of the target object with the projected image.

[0069] The modulation parameters include at least one of delay value, volume attenuation coefficient, phase offset, and sound field mode. The perceived position of the audio signal relative to the target object refers to the sound image position perceived by the target object based on the played audio signal. In one possible implementation, the updated modulation parameters of the multi-channel speaker module can make the audio signal output by the multi-channel speaker module match the perceived position of the target sound source within the projected image. For example, when a sound source exists in the projected image, the control module 01 can identify the pixel coordinates of the sound source in the projected image and convert the pixel coordinates into spatial coordinates based on the updated projection parameters; based on the spatial coordinates and the first position information, it calculates the delay value and phase offset of each speaker unit to synthesize a virtual point sound source at that spatial coordinate, so that the audio signal output by the multi-channel speaker module matches the perceived position of the target object with the sound source within the projected image.

[0070] For example, the updated projection parameters correspond to the spatial coordinates of the projected image. The control module 01 can determine the listening position of the target object relative to the robot based on the first position information, and determine the image position of the projected image relative to the robot based on the updated projection parameters; based on the difference between the listening position and the image position, it calculates the modulation parameters of each speaker unit, so that the virtual sound source position generated by the multi-channel speaker module using sound image synthesis technology coincides with the image position; the calculated modulation parameters are sent to the multi-channel speaker module to control the multi-channel speaker module to output audio signals according to the modulation parameters. In one possible implementation, control module 01 can calculate the first distance between the target object and the projected image, and the second distance between the robot and the projected image, based on the first position information of the target object, the spatial coordinates of the center of the projected image corresponding to the updated projection parameters, and the second position information of the robot. Based on a preset correspondence between distance and gain compensation (which can be pre-stored in the form of a parameter lookup table or database), a first gain compensation value is determined based on the first distance, and a second gain compensation value is determined based on the second distance. Then, based on the reference modulation parameters that make the orientation of the virtual sound source coincide with the orientation of the image, the first gain compensation value and the second gain compensation value are superimposed to obtain the updated modulation parameters, so that the audio signal output by the multi-channel speaker module can match the perceived orientation of the projected image while its loudness also matches the perceived visual distance of the projected image.

[0071] Optionally, before performing the above step S160, environmental acoustic parameters can be collected first through the robot's sound acquisition module; the modulation parameters can be compensated based on the environmental acoustic parameters to adapt to the acoustic characteristics of the current environment.

[0072] Optionally, in response to a stop tracking command for the target object, the robot can be controlled to stop moving, lock its current position, and maintain stable output of projection and audio signals.

[0073] In one possible implementation, control module 01 can continuously monitor changes in the projection position, changes in the target object's position, and fluctuations in ambient acoustic parameters, recalculate the modulation parameters of the multi-channel speaker module, and update the output. In another possible implementation, when the target object is detected to be outside the current sweet spot, control module 01 can control the robot body to make a small pose adjustment so that the target object is back within the sweet spot.

[0074] In the above projection method, by integrating the projection module and the multi-channel speaker module into the robot and responding to projection tracking commands to control the robot to follow the target object, continuous interaction between the user and the robot can be facilitated. By pre-establishing a spatial position binding relationship between the robot and the projected image, and dynamically adjusting the projection parameters based on this binding relationship as the robot follows the target object, the display state of the projected image on the projection carrier remains stable. At the same time, based on the real-time position information of the target object and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated, so that the perceived position of the audio signal output by the multi-channel speaker module is dynamically matched with the projected image, ensuring the audiovisual consistency of the projected image, thus improving the user's viewing experience.

[0075] In an exemplary embodiment, the steps described above for controlling the robot to follow the target object based on the first position information may include:

[0076] In response to the detection of multiple target objects, the robot is controlled to move to the middle area of ​​the multiple target objects based on the first position information of the multiple target objects.

[0077] For example, when a second target object (such as user B) appears in the scene, the control module 01 can simultaneously lock user A and user B through the visual perception module and obtain the first position information of user A and user B respectively; calculate the geometric center coordinates of the two based on the first position information of user A and user B, and determine the geometric center coordinates as the target position; control the robot to move to the target position through the robot's moving mechanism so that the robot is located in the middle area of ​​multiple target objects.

[0078] Furthermore, step S160 above may include:

[0079] Based on the first position information of multiple target objects and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated so that the audio signal output by the multi-channel speaker module matches the projected image for the perceived position of multiple target objects.

[0080] For example, the control module 01 can calculate the spatial geometric center of multiple target object listening points based on the first position information of users A and B, and use the position of this center relative to the robot as the average listening position; determine the image position of the projected image relative to the robot based on the updated projection parameters; calculate the modulation parameters of each speaker unit based on the difference between the average listening position and the image position; and send the calculated modulation parameters to the multi-channel speaker module to control the multi-channel speaker module to output audio signals according to the modulation parameters, so that both users A and B can perceive that the sound source comes from the projected image.

[0081] In one possible implementation, when there are multiple target objects, the control module 01 can first obtain the first position information of each target object, and calculate the audio output parameters corresponding to each target object based on the first position information and the updated projection parameters to obtain the individual modulation parameters corresponding to each target object; then, calculate the average value of the individual modulation parameters corresponding to multiple target objects, and use the average value as the modulation parameters of the multi-channel speaker module.

[0082] In another possible implementation, module 01 can be weighted according to the distance between each target object and the projected image. Specifically, the spatial distance between each target object and the center of the projected image, as well as the deviation angle of each target object relative to the center of the projected image, can be calculated separately. Based on the spatial distance and deviation angle, a weight value is assigned to each target object (target objects that are closer to the projected image and closer to the center of the projected image can be given a higher weight value). The modulation parameters of the multi-channel speaker module are obtained by weighted averaging based on the individual modulation parameters of each target object and its corresponding weight value.

[0083] In another possible implementation, when the number of target objects exceeds a preset threshold, the control module 01 can group the target objects according to the first position information of each target object, and divide the target objects that are spatially adjacent into the same group; for each group of target objects, the position of the target object located at the center of the group is selected as the virtual audience position of the group, or the geometric center point of the position coordinates of all target objects in the group is calculated as the virtual audience position of the group; based on the virtual audience positions of each group, the modulation parameters of the multi-channel speaker module are updated in the manner described above for calculating the modulation parameters of multiple target objects.

[0084] In this embodiment, when multiple target objects are detected, the robot is controlled to move to the middle area of ​​the multiple target objects, and the modulation parameters of the speaker module are updated uniformly based on the position information and projection parameters of the multiple target objects. This allows the projected image and audio perception to be adapted to the distribution range of the group of users at the same time, so that multiple target objects can obtain a consistent audio-visual experience.

[0085] In one exemplary embodiment, such as Figure 3 As shown, the above projection method may further include:

[0086] S150, in response to detecting that the projected image is about to exceed the projection carrier, updates the projection parameters so that the projected image is displayed at a smaller size on the projection carrier.

[0087] For example, before performing step S140 above, it can be detected whether the projected image is about to exceed the projection carrier. In one possible implementation, in response to detecting that the distance between the edge of the projected image and the edge of the projection carrier is less than a preset distance, the projection parameters can be updated so that the projected image is displayed smaller on the projection carrier. In another possible implementation, when the target object moves to the edge of the projection carrier, the robot follows the movement, causing the position of the projected image to shift. At this time, the projection parameters can be updated so that the projected image is displayed smaller on the projection carrier. Specifically, the display size of the projected image can be reduced by adjusting the image scaling parameters of the projection module, reducing the projection focal length, and / or adjusting the horizontal projection angle / vertical projection angle to avoid the image exceeding the boundary of the projection carrier.

[0088] In this embodiment, by automatically updating the projection parameters to reduce the display size of the projected image when it is detected that the projected image is about to exceed the projection carrier, the complete display of the projected image can be guaranteed within the limited projection carrier range, avoiding the loss of image content caused by the projected image exceeding the carrier boundary.

[0089] In one exemplary embodiment, such as Figure 4 As shown, the steps for updating projection parameters based on the robot's second position information and pre-established spatial position binding relationships may include:

[0090] A1, based on the first position information and / or the robot's second position information, determine the target projection carrier.

[0091] The target projection surface can be the original wall or a nearby wall.

[0092] For example, when control module 01 detects that the target object has moved to the edge area of ​​the original projection wall and the projected image is about to exceed the range of the original projection wall, it can scan the robot's surrounding environment through the lidar in the pose perception module and the visual perception module to identify the nearest available wall; the identified nearest available wall is then determined as the target projection carrier. Alternatively, when the robot moves to a new spatial area, it can select the wall closest to the robot and without obstruction from multiple projection carriers in the current space as the target projection carrier based on the robot's second position information and the environmental data collected by the visual perception module. Alternatively, when control module 01 detects that the target object is moving within a preset distance range of the original wall, it can maintain the original wall as the target projection carrier.

[0093] A2, based on the robot's second position information and the pre-established spatial position binding relationship, determines the target display position of the projected image on the target projection carrier.

[0094] The target display location can be the same location on the original wall or a suitable display location on a newly selected wall.

[0095] For example, the target spatial coordinates of the projected image corresponding to the projection parameters can be determined based on the robot's second position information and a pre-established spatial position binding relationship; based on the target spatial coordinates and the position information of the target projection carrier, the target display position of the projected image on the target projection carrier can be determined. Specifically, a ray can be constructed with the projection direction of the projection module as the ray direction, passing through the target spatial coordinates of the center of the projected image; the coordinates of the intersection point of this ray and the plane of the target projection carrier can be calculated; with this intersection point as the center, based on the size of the projected image and the wall boundary constraints, a flat, unobstructed, and boundary-free rectangular area can be determined on the target projection carrier as the target display position of the projected image.

[0096] A3 updates the projection parameters based on the target display position to project the image onto the target display position while keeping the display state of the projected image unchanged.

[0097] For example, when the target projection carrier is a nearby wall and its usable area is smaller than the original projection screen size, the target display position determined by the control module 01 can be the largest usable rectangular area within the boundary of the nearby wall. The control module 01 can recalculate and update the projection parameters based on the boundary coordinates of the target display position and the robot's second position information. For example, the projection focal length can be adjusted to reduce the size of the projection screen to fit the largest usable rectangular area; at the same time, the horizontal projection angle / vertical projection angle can be updated to project the reduced projection screen completely onto the target display position while maintaining the image clarity.

[0098] Alternatively, when the target projection surface is a nearby wall and its usable area is greater than or equal to the original projection screen size, the control module 01 can select the central area of ​​the nearby wall as the target display position. The control module 01 can calculate the straight-line distance, horizontal angle, and vertical angle between the target display position and the robot's second position information, and update the projection focal length, horizontal projection angle / vertical projection angle accordingly, so as to completely switch the projection screen from the original wall to the central area of ​​the nearby wall, while maintaining the size and clarity of the projection screen consistent with the original projection screen.

[0099] In this embodiment, by dynamically determining the target projection carrier and target display position during the robot's movement, and updating the projection parameters accordingly, it is possible to maintain the display state of the projected image unchanged in scenarios where the projection carrier is switched or the display position is changed, thus achieving continuous and stable projection across projection carriers.

[0100] In one exemplary embodiment, such as Figure 5 As shown, the above projection method may further include:

[0101] S1101, control the robot to move to the target position.

[0102] The target location can be a location in the current space where there are no obstacles and the projection field of view is unobstructed, or it can be any other location in the current space that is suitable for projection.

[0103] For example, the robot's pose perception module can be used for simultaneous localization and map building, while the robot's visual perception module scans the surrounding environment, identifies flat areas on the wall and avoids obstacles, and controls the robot to move autonomously to the vicinity of the flat area.

[0104] S1102, control the projection module to project onto the projection carrier, and obtain the projection parameters of the projection module.

[0105] For example, the projection module can project a target video onto a wall based on preset reference projection parameters and output the current projection parameters to the control module inline. These projection parameters include, but are not limited to, projection focal length, projection angle, and projection screen size. It is understood that the projection parameters are directly output inline by the projection module, without relying on visual perception functionality.

[0106] S1103: Obtain the third position information of the projection carrier, and establish a spatial coordinate system based on the third position information, projection parameters, and the robot's second position information.

[0107] The third position information of the projection carrier can be collected in real time by the visual perception module, or it can be pre-stored.

[0108] For example, the robot's visual perception module can collect three-dimensional point cloud data of the wall and calculate the wall's boundary coordinates, plane normal vectors, and other third position information. Then, based on the third position information of the projection carrier and the projection parameters, the spatial mapping relationship between the projected image and the projection carrier can be determined. Based on the spatial mapping relationship and the robot's second position information, a three-dimensional spatial coordinate system can be established with the robot's body center as the origin.

[0109] S1104, based on the spatial coordinate system and the preset projection adjustment strategy, determines the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters.

[0110] The preset projection adjustment strategies include, but are not limited to, keeping the display state of the projected image on the projection carrier unchanged; adjusting the projected image to shrink on the projection carrier when it is about to exceed the projection carrier; and maintaining the projected image on the target projection carrier when the robot moves.

[0111] For example, a mapping function can be constructed by pre-setting a projection adjustment strategy to characterize the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters. This mapping function can associate the robot's position changes, the adjustment of projection parameters and the spatial position of the projected image in real time.

[0112] In this embodiment, by controlling the robot to automatically complete the projection initialization and spatial calibration, a spatial coordinate system and position binding relationship are established between the projection carrier, the projection image and the robot, which can provide a spatial reference benchmark for subsequent dynamic adjustment and reduce manual intervention and debugging.

[0113] In one exemplary embodiment, such as Figure 6 As shown, the above projection method may further include:

[0114] S1105, locks the robot's posture.

[0115] For example, the robot's posture can be locked throughout the process of following the target object to maintain the stability of the robot body during movement.

[0116] In one possible implementation, the step of establishing a spatial coordinate system based on the third position information of the projection carrier, projection parameters, and the second position information of the robot may include: establishing a spatial coordinate system based on the third position information of the projection carrier, projection parameters, the second position information of the robot, and attitude information. Specifically, the spatial mapping relationship between the projected image and the projection carrier can be determined based on the third position information and projection parameters of the projection carrier; based on the spatial mapping relationship, the second position information of the robot, and attitude information, a three-dimensional spatial coordinate system is established with the robot's center as the origin and the robot's current orientation as the positive Y-axis. The step of determining the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters based on the spatial coordinate system and a preset projection adjustment strategy may include: establishing a mapping function between the three-dimensional coordinates of the center of the projected image and the robot's pose parameters and projection parameters. The robot's pose parameters include the second position information and attitude information.

[0117] Furthermore, step S160 above may include:

[0118] S1601, based on the pose perception module, acquires the robot's posture change information.

[0119] For example, the robot's three-axis angular velocity and acceleration data can be collected in real time by the inertial sensors in the robot's pose perception module; at the same time, the distance change between the robot and the projection wall can be measured in real time by the lidar and ranging sensors in the pose perception module, and the robot's attitude change information (such as pitch angle offset, yaw angle offset, etc.) can be calculated.

[0120] S1602, based on attitude change information, first position information and updated projection parameters, update the modulation parameters of the multi-channel speaker module.

[0121] For example, the acoustic-image perception deviation caused by the robot's posture shift can be calculated based on the posture change information; the acoustic-image perception deviation can be superimposed and compensated with the reference modulation parameters calculated based on the first position information and the updated projection parameters to generate the corrected modulation parameters; the corrected modulation parameters can be sent to the multi-channel speaker module to update the delay value, volume attenuation coefficient and phase shift of each speaker to compensate for the influence of the robot's posture change on acoustic-image localization, so that the sound source location perceived by the target object continuously matches the center of the projected image.

[0122] In this embodiment, by locking the robot's posture to maintain stability during projection and sensing posture change information in real time, and using it as the basis for adjusting audio modulation parameters, the sound-image misalignment caused by robot body shaking can be reduced.

[0123] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0124] Based on the same inventive concept, this application also provides a projection device for implementing the projection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more projection device embodiments provided below can be found in the limitations of the projection method described above, and will not be repeated here.

[0125] In one exemplary embodiment, such as Figure 7 As shown, a projection device 700 is provided, including: a robot control module 701, a projection parameter update module 702, and an audio parameter update module 703, wherein:

[0126] The robot control module 701 is used to respond to a projection tracking command for a target object, acquire the first position information of the target object, and control the robot to follow the target object based on the first position information;

[0127] The projection parameter update module 702 is used to update the projection parameters based on the robot's second position information and the pre-established spatial position binding relationship during the movement of following the target object, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters.

[0128] The audio parameter update module 703 is used to update the modulation parameters of the multi-channel speaker module based on the first position information and the updated projection parameters, so that the audio signal output by the multi-channel speaker module matches the perceived position of the target object with the projected image.

[0129] In one embodiment, the robot control module 701 may include:

[0130] The first robot control submodule is used to control the robot to move to the middle area of ​​the multiple target objects based on the first position information of the multiple target objects in response to the detection of multiple target objects.

[0131] Furthermore, the aforementioned audio parameter update module 703 may include:

[0132] The first audio parameter update submodule is used to update the modulation parameters of the multi-channel speaker module based on the first position information of multiple target objects and the updated projection parameters, so that the audio signal output by the multi-channel speaker module matches the projected image for the perceived position of multiple target objects.

[0133] In one embodiment, the projection parameter update module 702 described above can also be used for:

[0134] In response to the detection that the projected image is about to exceed the projection medium, the projection parameters are updated to make the projected image appear smaller on the projection medium.

[0135] In one embodiment, the projection parameter update module 702 may include:

[0136] The projection carrier determination submodule is used to determine the target projection carrier based on the first position information and / or the robot's second position information.

[0137] The display position determination submodule is used to determine the target display position of the projected image on the target projection carrier based on the robot's second position information and the pre-established spatial position binding relationship.

[0138] The projection parameter update submodule is used to update the projection parameters based on the target display position, so as to project the image onto the target display position and keep the display state of the image unchanged.

[0139] In one embodiment, the projection device 700 may further include an initialization module, which may include:

[0140] The second robot control submodule is used to control the robot to move to the target position.

[0141] The projection submodule is used to control the projection module to project onto the projection carrier and to obtain the projection parameters of the projection module.

[0142] The coordinate system establishment submodule is used to obtain the third position information of the projection carrier and establish a spatial coordinate system based on the third position information, projection parameters and the robot's second position information.

[0143] The spatial position binding submodule is used to determine the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters based on the spatial coordinate system and the preset projection adjustment strategy.

[0144] In one embodiment, the robot control module 701 is further configured to lock the robot's posture; further, the audio parameter update module 703 may include:

[0145] The pose acquisition submodule is used to acquire the robot's pose change information based on the pose perception module.

[0146] The second audio parameter update submodule is used to update the modulation parameters of the multi-channel speaker module based on the posture change information, the first position information, and the updated projection parameters.

[0147] Each module in the aforementioned projection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0148] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0149] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0150] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A projection method, characterized in that, A control module for a robot, the robot further comprising a projection module and a multi-channel speaker module, the projection module being used to project onto a projection carrier based on projection parameters to form a projected image on the projection carrier, the method comprising: In response to a projection tracking command for a target object, the robot acquires the first position information of the target object and controls the robot to follow the target object based on the first position information. During the movement following the target object, the projection parameters are updated based on the robot's second position information and the pre-established spatial position binding relationship, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters; Based on the first location information and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated so that the audio signal output by the multi-channel speaker module matches the projected image with the perceived position of the target object.

2. The method according to claim 1, characterized in that, The step of controlling the robot to follow the target object based on the first location information includes: In response to the detection of multiple target objects, the robot is controlled to move to the middle area of ​​the multiple target objects based on the first position information of the multiple target objects; Updating the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters includes: Based on the first position information of the multiple target objects and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated so that the audio signal output by the multi-channel speaker module matches the projected image for the perceived position of the multiple target objects.

3. The method according to claim 1, characterized in that, The method further includes: In response to detecting that the projected image is about to exceed the projection carrier, the projection parameters are updated so that the projected image is displayed at a smaller size on the projection carrier.

4. The method according to any one of claims 1 to 3, characterized in that, The updating of the projection parameters based on the robot's second position information and the pre-established spatial position binding relationship includes: Based on the first position information and / or the second position information of the robot, determine the target projection carrier; Based on the robot’s second position information and the pre-established spatial position binding relationship, the target display position of the projected image on the target projection carrier is determined. Based on the target display position, the projection parameters are updated to project the image onto the target display position while keeping the display state of the image unchanged.

5. The method according to any one of claims 1 to 3, characterized in that, Prior to the response to the projection tracking command for the target object, the following is included: Control the robot to move to the target location; Control the projection module to project onto the projection carrier, and obtain the projection parameters of the projection module; Obtain the third position information of the projection carrier, and establish a spatial coordinate system based on the third position information, the projection parameters, and the second position information of the robot; Based on the spatial coordinate system and the preset projection adjustment strategy, the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters is determined.

6. The method according to claim 5, characterized in that, The robot also includes a pose perception module, which, after controlling the robot to move to the target position, includes: Lock the robot's posture; Updating the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters includes: The robot's posture change information is obtained based on the posture perception module; Based on the posture change information, the first position information, and the updated projection parameters, the modulation parameters of the multi-channel speaker module are updated.

7. A projection device, characterized in that, A control module for a robot, the robot further comprising a projection module and a multi-channel speaker module, the projection module being used to project onto a projection carrier based on projection parameters to form a projected image on the projection carrier, the device comprising: The robot control module is used to respond to a projection tracking command for a target object, acquire the first position information of the target object, and control the robot to follow the target object based on the first position information; The projection parameter update module is used to update the projection parameters based on the robot's second position information and a pre-established spatial position binding relationship during the movement of following the target object, so that the display state of the projected image on the projection carrier remains unchanged; wherein, the pre-established spatial position binding relationship includes the spatial position binding relationship between the robot and the projected image corresponding to the projection parameters; An audio parameter update module is used to update the modulation parameters of the multi-channel speaker module based on the first location information and the updated projection parameters, so that the audio signal output by the multi-channel speaker module matches the projected image with respect to the perceived position of the target object.

8. A robot, characterized in that, The robot includes a control module, a projection module, and a multi-channel speaker module; The projection module is used to project onto the projection carrier based on projection parameters to form a projected image on the projection carrier. The control module is used to implement the steps of the method according to any one of claims 1 to 6 when executing a computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images