A projection interaction method, device and system

Abstract

The application provides a projection interaction method, device and system. The method comprises the following steps: controlling a projection device to project a display picture to a projection medium; detecting a detection object in a detection range by a position detection module, and determining position information of the detection object; determining mapping coordinate parameters according to the position information, wherein the mapping coordinate parameters comprise mapping coordinates of the detection object in the display picture and detection object motion state information, and the detection object motion state information is used for representing a motion state of the detection object; determining an interaction operation according to the mapping coordinate parameters, and controlling the display picture to respond to the interaction operation. By converting the position information of the detection object into the mapping coordinate parameters in the display picture and combining the detection object motion state information to determine the interaction operation, the user can interact with the display picture through the detection object in front of the projection medium.

Description

Technical Field

[0001] This application relates to the field of display, and in particular to a projection interaction method, apparatus and system. Background Technology

[0002] With the development of in-vehicle cabin entertainment needs, large rear-seat entertainment screens are gradually being applied to mid-to-high-end vehicles. In order to provide a larger display screen, some vehicles have rear-seat displays behind the first row of seats, thus forming a rear-seat entertainment display area in the vehicle.

[0003] Currently, in-vehicle rear-seat display interaction methods mainly include ceiling-mounted touchscreens, voice control, second-row screen control, and projection screen displays. Ceiling-mounted screens support touch operation, but their display area is relatively small due to screen size and installation location limitations. While voice control and second-row screen control can perform some operations, users need to indirectly control the projected image through voice commands or other screens, making the interaction less direct.

[0004] For in-vehicle projectors with screens, the projection screen offers a large display area, making it suitable for rear-seat entertainment scenarios. However, traditional projection screens typically only serve as a display surface and cannot directly support touch operation. Integrating capacitive touch or infrared touch with the screen may be affected by factors such as screen roll-up, structural layout, durability, and installation space. Summary of the Invention

[0005] This application provides a projection interaction method, device, and system that can acquire the position and motion state information of the detected object through a position detection module, thereby identifying interactive operations and controlling the response of the projected image to achieve more accurate projection interaction.

[0006] Firstly, this application provides a projection interaction method, the method comprising: controlling a projection device to project a display screen onto a projection medium; detecting a detection object within a detection range using a position detection module, and determining the position information of the detection object; determining mapping coordinate parameters based on the position information, the mapping coordinate parameters including the mapping coordinates of the detection object in the display screen and motion state information of the detection object, the motion state information representing the motion state of the detection object; determining an interaction operation based on the mapping coordinate parameters, and controlling the display screen to respond to the interaction operation. By converting the position information of the detection object into mapping coordinate parameters in the display screen, and combining this with the motion state information of the detection object to determine the interaction operation, a user can interact with the display screen through a detection object in front of the projection medium.

[0007] In one possible implementation, determining the location information of the detected object specifically includes: determining the distance between the detected object and the location detection module; and determining the location information of the detected object based on the distance between the detected object and the location detection module. By using the distance between the detected object and the location detection module to determine the location information of the detected object, a detection basis can be provided for subsequent location mapping and interactive operation recognition.

[0008] In one possible implementation, the position detection module further includes a driving module, which controls the position detection module to scan within the detection range. The driving module determines the position information of the detected object based on the distance between the detected object and the position detection module, specifically by determining the position information of the detected object based on the distance and the scanning angle of the position detection module. By combining the distance and the scanning angle, the position of the detected object within the detection range can be determined, facilitating the determination of the position of the detected object relative to the projection medium.

[0009] In one possible implementation, determining the location information of the detected object includes: generating a coordinate point cloud based on the detection points corresponding to the detected object; and filtering the coordinate point cloud to determine the location information of the detected object. By processing the coordinate point cloud generated from the detection points, the impact of errors at individual detection points on the location determination result can be reduced.

[0010] In one possible implementation, filtering the coordinate point cloud includes processing the coordinate point cloud to remove at least one of background points, invalid detection points, and noise points from the projection medium. By removing background points, invalid detection points, and noise points, the influence of the detection results corresponding to the projection medium, environmental structure, or detection errors on the determination of the location of the detected object can be reduced.

[0011] In one possible implementation, determining the mapping coordinate parameters based on the position information includes: converting the position information of the detected object into mapping coordinate parameters based on a preset coordinate mapping relationship; wherein the preset coordinate mapping relationship is used to characterize the correspondence between the position information of the detected object and the mapped coordinates in the display screen. Through the preset coordinate mapping relationship, the position of the detected object in front of the projection medium can correspond to its display position in the display screen.

[0012] In one possible implementation, determining the interactive operation based on the mapped coordinate parameters includes: determining a first action based on the mapped coordinate parameters, the first action including at least one of a click action, a double-click action, a long-press action, a swipe action, a drag action, and a multi-finger action; and determining the interactive operation based on the first action. By first determining the first action and then determining the interactive operation based on the first action, corresponding interactive processing can be performed according to different action types.

[0013] In one possible implementation, controlling the display screen to respond to interactive operations includes: controlling the display screen to execute a first response operation based on a first action, the first response operation including at least one of interface adjustment, target selection, playback adjustment, volume adjustment, and brightness adjustment. By executing the corresponding first response operation based on the first action, the display screen can be updated according to the user's operation.

[0014] In one possible implementation, the method further includes displaying a cursor on the display screen based on mapped coordinate parameters, wherein the cursor is used to indicate the current position of the detected object on the display screen. By displaying the cursor, the user can be prompted with the corresponding position of the detected object on the display screen.

[0015] In one possible implementation, the detection object includes a user's finger or a tool used to point at the projection medium. By using the user's finger and the tool used to point at the projection medium as detection objects, different interaction methods can be adapted.

[0016] In one possible implementation, the detection module detects objects within the detection range, including: determining the distance between the detection object and the projection medium; determining whether the distance between the detection object and the projection medium is within a set distance range; and if it is within the set distance range, determining the position information of the detection object. By determining whether the distance between the detection object and the projection medium is within the set distance range, irrelevant objects far from the projection medium can be avoided from being used for interactive operation determination.

[0017] In one possible implementation, determining the mapping coordinate parameters based on location information includes: when there are multiple detected objects, determining the mapping coordinate parameters corresponding to each of the multiple detected objects based on their respective location information. By determining the mapping coordinate parameters corresponding to multiple detected objects separately, multi-finger operations or multi-object operations can be supported.

[0018] In one possible implementation, the position detection module is disposed on the transverse support beam used for winding the projection medium. By disposing the position detection module on the transverse support beam, a relatively fixed positional relationship can be maintained between the position detection module and the projection medium, facilitating the determination of the position of the object being detected relative to the projection medium.

[0019] Secondly, this application provides a projection interaction device, which includes a projection module, a position determination module, a coordinate determination module, and an interaction module. The projection module controls a projection device to project a display image onto a projection medium; the position determination module detects an object within a detection range using a position detection module and determines the object's position information; the coordinate determination module determines mapping coordinate parameters based on the position information, including the object's mapping coordinates on the display screen and the object's motion state information, which represents the object's motion state; the interaction module determines an interaction operation based on the mapping coordinate parameters and controls the display screen to respond to the interaction operation. Through the cooperation of the projection module, position determination module, coordinate determination module, and interaction module, projection display, position detection, coordinate mapping, and interactive response can be achieved.

[0020] Thirdly, this application provides a projection interaction system, which includes a projection device, a projection medium, a position detection module, and a projection interaction device as described in the first aspect. Through the cooperation between the projection device, the projection medium, the position detection module, and the projection interaction device, interactive operations on the displayed image can be realized in front of the projection medium.

[0021] Fourthly, this application provides a vehicle that includes the projection interaction system provided in the third aspect.

[0022] Fifthly, this application provides a projection interaction device, which includes a processor and a memory coupled together. The memory stores computer programs or instructions, and the processor executes the computer programs or instructions to implement the projection interaction method of the first aspect. The projection interaction method can be implemented by the processor executing the computer programs or instructions in the memory.

[0023] In a sixth aspect, this application provides a computer-readable storage medium storing a computer program or instructions, which, when executed, implements the projection interaction method of the first aspect.

[0024] In a seventh aspect, this application provides a computer program product, which includes instructions that, when executed, implement the projection interaction method of the first aspect. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a projection interaction system structure; Figure 2 This is a schematic diagram of the structure of a lidar. Figure 3 A schematic diagram of the projection interaction method steps provided in this application; Figure 4A schematic diagram of one arrangement of the projection medium is shown; Figure 5 A diagram illustrating the setting of a distance range; Figure 6 This is a schematic diagram of the layout of a position detection module; Figure 7 A schematic diagram of a detection method for multiple detection objects; Figure 8 Another schematic diagram for detecting multiple objects; Figure 9 This is a schematic diagram illustrating how to determine the location information of a detected object based on distance and scanning angle. Figure 10 This is a schematic diagram showing the correspondence between the position of a detection object and the position of the display cursor. Figure 11 A possible structural schematic diagram of a projection interaction device provided in this application is shown; Figure 12 This is a schematic diagram of another possible structure for a projection interactive device. Detailed Implementation

[0026] The relevant technical solutions are described below with reference to the accompanying drawings. It should be understood that the following content is used to illustrate the implementation of this application and is not intended to limit the scope of protection. Where there is no conflict, technical features from different implementations can be combined with each other.

[0027] To facilitate understanding, the following text will first explain some of the terms used.

[0028] 1. Projection equipment, which can be a vehicle-mounted projection optical engine, a mini projector, or other devices capable of converting a display image into an optical projection image. The projection equipment can connect to vehicle infotainment systems, cockpit controllers, entertainment systems, or other display control devices, and generate a projection beam based on input video data, desktop interfaces, or application interfaces.

[0029] II. The projection medium can be a projection screen, projection film, rollable display surface, or other structure capable of supporting the projected image. In a vehicle setting, the projection medium can be placed behind the first row of seats or in other suitable viewing locations within the vehicle. The projection medium can be a flexible screen that can be unfolded and rolled up using horizontal support beams or a roll-up assembly.

[0030] 3. Position detection module, used to detect objects within the detection range and output data related to the position of the detected objects. The position detection module may include one or more of the following: a direct time-of-flight sensor, a drive module, a controller, a signal processing circuit, or a communication interface.

[0031] IV. Direct Time of Flight (DTOF) sensor: It can determine the distance between the object being detected and the sensor by emitting detection light and receiving reflected light. DTOF can output distance data or combine it with the scanning angle to generate detection points in the detection area.

[0032] V. Coordinate point cloud can be understood as a data set composed of multiple detection points. Each detection point can be recorded as distance, angle, intensity, timestamp, or a converted attribute value. Coordinate point cloud can be used to represent the detected object, background structure, and noise points in front of the projection medium.

[0033] The preceding text introduced some of the terms used in this application. The following text introduces the possible application scenarios of this application.

[0034] This projection interaction method can be applied to rear-seat entertainment scenarios in vehicles. For example, a projection screen can be installed behind the first-row seats, and a projection device can be installed above the second or third-row seats. The vehicle's infotainment system or cockpit controller provides video, desktop interfaces, or application interfaces to the projection device, which then projects the display onto the projection screen. Rear-seat passengers can then perform actions such as clicking or swiping in front of the screen using their fingers.

[0035] In this scenario, users do not need to control the projected image through voice commands or dual-screen mirroring; instead, they can interact directly with the projection screen. The position detection module detects the position of the user's finger in front of the screen and outputs detection data. Based on the detection data, the finger position is determined and mapped to the display screen coordinate system. Then, based on the mapped coordinates and the action type, the projection device is controlled to update the display screen. In this way, the projection screen can serve as both a large-size display medium and an interaction entry point.

[0036] This method can also be applied to multi-occupant scenarios inside a vehicle. For example, passengers on the left side of the second row and the right side of the second row can both operate in front of the projection screen. The position detection module can detect multiple objects simultaneously and determine the mapping coordinates of each object on the display screen. Based on the position of different objects, it can determine whether to perform split-screen control, multi-finger zoom, or multi-person collaborative operation.

[0037] This method can also be applied to scenarios requiring hover touch control. Since the projection medium may be a flexible screen, frequent direct pressing may cause the screen to become dirty, deformed, or scratched. By setting a predetermined distance range between the detection object and the projection medium, interaction can be triggered when the detection object is close to but does not necessarily touch the projection medium, thereby reducing contact with the surface of the projection medium.

[0038] Beyond rear-seat entertainment systems in vehicles, this method can also be applied to conferences, education, exhibition hall displays, or other interactive projection scenarios. As long as the system contains a projection device, projection medium, and a position detection module, the position of the detected object can be mapped to the display screen coordinate system in a similar manner, and the projection device can be controlled to update the displayed image.

[0039] See Figure 1 The diagram illustrates a projection interaction system. This system may include a projection interaction device, a projection equipment, a projection medium, and a position detection module. The projection interaction device can provide a display screen or projection control commands to the projection equipment, causing the projection equipment to project the display screen onto the projection medium. Optionally, the display screen provided by the projection equipment may originate from the vehicle's infotainment system. The position detection module detects objects in front of the projection medium and provides the detection data to the projection interaction device. The projection interaction device determines the mapping coordinate parameters and interaction operations based on the detection data and controls the projection equipment to update the display screen.

[0040] The position detection module provided in this application embodiment can be a direct time-of-flight sensor, which can be a lidar. For example, see [link to relevant documentation]. Figure 2 The diagram shows the structure of a lidar system. The lidar's hardware architecture mainly includes a laser emitting section 100, a laser receiving section 200, and a signal processing system 300. The laser emitting section 100 includes an excitation source, a laser, and a transmitting optical system. The laser receiving section 200 includes a receiving optical system and a detector. Its specific composition and working principle are as follows: An excitation source drives a laser to emit a laser beam (or laser pulse), which is then emitted outward through an emitting optical system. When the laser beam emitted from the lidar encounters a target object, it interacts with the object to form a reflected / scattered echo beam. This echo beam is collected by a receiving optical system and received by a detector. The optical signal is converted into an electrical signal via an analog-to-digital converter (ADC), and then processed by an analog front-end before being transmitted to a signal processing system 300. The signal processing system 300 processes the received signal to obtain information such as the target object's distance, velocity, and azimuth. Furthermore, it can acquire information such as the target's surface morphology and physical properties to build an object model. The detector is typically a photodetector, which converts the received optical signal into an electrical signal. This electrical signal is usually an analog signal, while the signal processing system 300 typically processes digital signals, such as a digital signal processor (DSP). Therefore, the analog electrical signal is converted into a digital signal via an ADC and provided to the signal processing system 300. Additionally, the electrical signal can be amplified, and then converted back into a digital signal via an ADC before being provided to the signal processing system 300. The signal processing system 300 includes signal processing circuitry for processing digital signals to obtain information such as the target object's distance, velocity, and azimuth, and further establishes an object model. The lidar also includes control circuitry, such as a control unit for controlling the excitation source and a control unit for controlling the scanning drive circuit. These two control units can be integrated or set up independently. Furthermore, the signal processing circuitry and control circuitry can also be integrated or set up independently.

[0041] Furthermore, the lidar may also include a scanning section 400. The laser beam emitted by the laser emitting section 100, under the action of the scanning section 400, scans the laser beam on a plane to generate real-time planar image information. The scanning section 400 mainly includes a scanning mechanism and a scanning drive circuit. The scanning drive circuit drives the scanning mechanism to operate, and the laser beam, under the action of the scanning mechanism, achieves the transformation from a line to a plane.

[0042] Currently, the demand for large rear-seat entertainment screens in vehicles is growing. While projection screens offer a large display area, traditional solutions struggle with direct touch control. Integrating capacitive or infrared touchscreens involves limitations such as roll-up mechanisms, space constraints, and durability, resulting in less direct interaction. To address these issues, this application proposes a projection interaction method that uses a position detection module to acquire the position and motion status information of the detected object. Based on this information, it identifies interactive operations and controls the projection screen response, achieving more precise projection interaction.

[0043] The following is combined with Figure 3 This application describes a projection interaction method. Figure 3 As shown, the method may include the following steps.

[0044] Step S301: Control the projection device to project the display image onto the projection medium.

[0045] In this step, the projection device can be used to project the display image onto the projection medium using optical projection, so that the projection medium forms a display area that can be viewed and interacted with by the user. The projection device can receive video data, image data, or interface data output from the display source, and generate a projection beam based on the data. After the projection beam shines on the projection medium, a corresponding display image can be formed on the projection medium.

[0046] The display screen can be understood as the visual content that needs to be shown to the user. The display screen can include video playback interface, desktop interface, application interface, game interface, navigation interface, vehicle control interface, conference presentation interface, or other interfaces that can be displayed through projection. The display screen can also include images, text, icons, buttons, lists, progress bars, menus, control controls, or other display objects.

[0047] The source of the displayed content can be various. For example, the displayed content can come from the vehicle's infotainment system, cockpit controller, entertainment head unit, rear-seat entertainment head unit, mobile terminal projection content, external playback devices, or cloud application content. The vehicle's infotainment system can provide application interfaces or entertainment content within the vehicle's cockpit; the cockpit controller can provide display content related to the vehicle's cockpit functions; the entertainment head unit can provide entertainment interfaces such as videos, music, and games; and mobile terminal projection content can come from mobile phones, tablets, or other portable devices.

[0048] Projection devices can be connected directly to a display source or through an intermediate control device. For example, a projection device can receive display data output from a display source via a high-definition multimedia interface, universal serial bus, automotive Ethernet, wireless LAN, Bluetooth, or other communication methods. After receiving the display data, the projection device can generate a corresponding projected image and project that image onto a projection medium.

[0049] The projection device can be a vehicle-mounted projection device or a projection device in other scenarios. Taking a vehicle-mounted projection device as an example, the vehicle-mounted projection device can be installed on the roof of the vehicle, above the seats, on the interior trim, on the dashboard, near the seat back, or in other suitable projection locations. The installation location of the projection device can be determined based on factors such as the location of the projection medium, the interior space of the vehicle, the viewing angle of the occupants, and the wiring harness layout.

[0050] Projection media can be understood as the medium used to carry the projected image. Projection media can be projection screens, projection displays, vehicle interior walls, vehicle interior partitions, seat back surfaces, window projection films, flexible, unfoldable screens, or other structures capable of receiving the projection beam. Projection media can be fixed, or they can be unfoldable, rollable, or foldable. See also... Figure 4 As shown, a schematic diagram of one arrangement of the projection medium is illustrated.

[0051] In in-vehicle rear-seat entertainment scenarios, the projection medium can be an in-vehicle projection screen. Once unfolded, the screen creates a large projection display area inside the vehicle. The projection equipment projects the displayed image onto the screen, allowing second- or third-row passengers to view the image and interact with it. The projection screen can be located behind the first-row seats, or it can be placed on the vehicle's roof, seatbacks, interior partition structure, or other suitable viewing locations for rear passengers.

[0052] When the projection medium is a retractable screen, the projection display can be started after the screen is fully unfolded, thus reducing image distortion caused by projection when the screen is not fully unfolded. The projection interaction system can obtain the screen unfolding status or receive an unfolding completion signal sent by the screen controller. After confirming that the projection medium is in a projectable state, it controls the projection device to output the display image.

[0053] The projection medium can also be a fixed projection surface. For example, in conference rooms, exhibition halls, or educational settings, the projection medium can be a wall, a fixed screen, or a display substrate. In this case, the projection device can be positioned in front of, above, or to the side of the projection medium, allowing users to face the projection medium to view the displayed image and interact with it using their fingers or pointing tools.

[0054] As one possible implementation, the detection object can be a user's finger, a stylus, a pointer, or other tool used to point at the projection medium. The detection object does not need to be in contact with the projection medium; as long as the detection object enters the detection range of the position detection module and can be detected by the position detection module, it can be used as the object for subsequently determining the interaction position.

[0055] Controlling a projection device to project a display image onto a projection medium can include processes such as starting the projection device, receiving display data, adjusting projection parameters, and outputting the projection beam. Projection parameters can include brightness, focal length, projection ratio, projection area, calibration parameters, image scaling parameters, image offset parameters, or other parameters that affect the projection display effect.

[0056] If the projection equipment is not perfectly aligned with the projection medium, the projected image may be distorted, offset, or missized. Therefore, when controlling the projection equipment to display the image, projection correction can be performed based on the position of the projection medium, the position of the projection equipment, the projection distance, and the size of the projection area.

[0057] Projection correction can include at least one of the following: geometric correction, boundary correction, image scaling, image panning, or focal length adjustment. For example, when a projection device is positioned above a projection medium and projects downwards at a certain angle, the projected image may form a shape that is narrower at the top and wider at the bottom, or vice versa. In this case, geometric correction can be used to make the displayed image on the projection medium approximate a rectangle.

[0058] Projection correction can also be related to subsequent coordinate mapping. Since the position of the detected object needs to be mapped to its coordinate position on the display screen, the position and size of the projected image on the projection medium will affect the mapping relationship. If the projection area changes, the coordinate mapping relationship can be updated accordingly, ensuring that the position of the detected object in front of the projection medium corresponds to the correct position on the display screen.

[0059] The effective display area can be determined based on the size of the projection medium, the installation location of the projection equipment, the resolution of the display screen, and the user's viewing needs. The effective display area can be smaller than the physical area of ​​the projection medium, or it can essentially cover the display area of ​​the projection medium. For example, when there are curled edges, support structures, or non-display borders at the edges of a vehicle screen, the display image can be restricted to the central area of ​​the screen to reduce the impact of edge distortion on display and interaction.

[0060] In vehicle usage scenarios, controlling the projection display can also be related to the vehicle's status. For example, after the vehicle starts, the projection device can be allowed to enter standby mode; after the user activates the rear entertainment function, the projection device can be controlled to output the display; when the projection medium is rolled up, the projection display can be stopped; when the vehicle is in certain safety-restricted states, the display or interaction of some vehicle control interfaces can be restricted.

[0061] In mobile terminal screen mirroring scenarios, the displayed image can come from a mobile phone or tablet. The mobile terminal can send videos, application interfaces, or desktop content to the vehicle's infotainment system or a projection device, which then projects this content onto a projection medium. In this case, the projection medium can serve as the display carrier for the mobile terminal's content, and subsequent interactive control of the projected image can be achieved through a position detection module.

[0062] In the exhibition scenario, the displayed screen can be a product introduction interface, an interactive demonstration interface, or multimedia content. The projection medium can be set on an exhibition wall, exhibition stand, or screen. After the projection device projects the display screen onto the projection medium, visitors can perform operations such as selection, sliding, or zooming in front of the projection medium. In this scenario, the projection medium does not need to integrate a traditional touch layer; interaction can still be achieved through subsequent position detection and coordinate mapping.

[0063] Projection devices can form a fixed correspondence with projection media. For example, the projection device and projection media can be installed in a vehicle as part of the same projection interaction system. The projection direction, projection distance, and projection area of ​​the projection device are calibrated at the factory. During vehicle use, preset projection parameters can be directly invoked to project the display image onto the preset area.

[0064] In other implementations, the positional relationship between the projection device and the projection medium may change. For example, the projection medium can be adjusted in height, the projection device can be angled, or the projection medium can switch positions depending on the usage scenario. In this case, projection area confirmation can be performed before projection begins, or projection parameters can be adjusted based on sensor data, calibration parameters, and user settings.

[0065] When the projection medium is a flexible screen, the screen may be slightly bent, shake, or have an uneven surface after it is unfolded. When controlling the projection device to project the image, a certain display deviation can be allowed, and some deviations can be compensated through projection correction and coordinate mapping.

[0066] After the display screen has been projected onto the projection medium, the user can determine the position to be operated based on the visible content on the projection medium. Subsequently, the position detection module detects the position of the user's finger or pointing tool and maps it to the coordinates on the display screen to determine the click, swipe, long press or other interactive operation.

[0067] Step S302: The location detection module detects the detection object within the detection range and determines the location information of the detection object.

[0068] The detection range can be understood as the spatial area in which the position detection module can effectively obtain detection data. This detection range can cover the area in front of the projection medium or a part of the projection medium.

[0069] In vehicle-mounted projection screen scenarios, the detection range can cover an area within a certain distance in front of the screen, allowing detection when a user's finger or pointing tool approaches the screen. In conference, education, or exhibition hall scenarios, the detection range can cover the area in front of the projected image, allowing detection when a user interacts with the projected image using their finger or pointing tool.

[0070] The position detection module can be placed near the projection medium. "Near" here means that the position detection module and the projection medium have a relatively fixed positional relationship, allowing the detection data output by the module to determine the position of the detected object relative to the projection medium or display screen. The position detection module can be placed above, below, to the side, or behind the projection medium, or it can be placed on a structure connected to or adjacent to the projection medium.

[0071] As one possible implementation, the detection module detects the detection object within the detection range, specifically including: determining the distance between the detection object and the projection medium, judging whether the distance between the detection object and the projection medium is within a set distance range, and if it is within the set distance range, determining the position information of the detection object.

[0072] The distance between the object being detected and the projection medium can be determined based on the detection data obtained by the position detection module. When the position detection module can directly obtain the distance between the object and the projection medium, this distance can be used as the basis for judgment. (See also...) Figure 5 The diagram shown is a schematic of setting a distance range. After the position detection module obtains the distance of the detection object relative to itself, it can determine the distance between the detection object and the projection medium by combining the relative positional relationship between the position detection module and the projection medium.

[0073] Setting a distance range can include a lower distance limit and an upper distance limit. The lower distance limit can be used to prevent the detection results from being affected when the detected object is too close to the projection medium, and it can also be used to distinguish between contact and non-contact states. The upper distance limit can be used to limit the range of positions that can be used for interaction, and to prevent irrelevant interactive objects that are far away from the projection medium from being detected.

[0074] For example, if the distance between the object being detected and the projection medium is greater than or equal to the lower distance limit and less than or equal to the upper distance limit, it can be determined that the object is within the set distance range, and the position information of the object can be determined. If the distance between the object being detected and the projection medium is less than the lower distance limit or greater than the upper distance limit, the position information of the object can be left undetermined, or the corresponding position information can be marked as invalid.

[0075] The distance range can be set based on the installation location of the projection medium, the detection capability of the position detection module, the size of the projected image, and interaction requirements. When the projection medium allows for contact operation, the lower distance limit can be close to the plane of the projection medium. When operation is primarily suspended in the air, the lower distance limit can be greater than zero to maintain a set distance between the detected object and the projection medium.

[0076] For distance values ​​obtained from continuous detection, the results from multiple detection cycles can be combined for judgment. If the detected object is within the set distance range in multiple detection cycles, the position of the detected object can be determined to be valid. If the detected object only enters the set distance range in a single detection cycle, or if the distance value changes significantly, the position information of the detected object can be temporarily uncertain to reduce misjudgment of position caused by short-term detection errors.

[0077] When a detected object enters the set distance range from outside the set distance range, its position information can be determined; when the detected object leaves the set distance range, its position information can be stopped from being updated. This allows subsequent interaction positions to be determined based on detected objects located within an effective distance range in front of the projection medium.

[0078] See Figure 6 The diagram shows a schematic of a position detection module arrangement. In a vehicle-mounted projection screen scenario, the position detection module is positioned on a transverse support beam used for rolling up the projection medium. The transverse support beam can support the unfolding of the projection medium or serve as part of the projection medium rolling mechanism. Placing the position detection module on the transverse support beam allows the module to maintain a relatively stable relative position with the projection medium, facilitating subsequent determination of the detected object's position in front of the projection medium based on detection data.

[0079] The transverse support beam used for rewinding the projection medium can include a roller, housing, motor, bracket, or structure for storing the projection medium. For retractable screens, the rewind assembly is typically located on one side of the screen. Placing the position detection module on the rewind assembly reduces modifications to the screen fabric and facilitates the use of space within the rewind assembly for arranging sensors, wiring harnesses, and control circuitry.

[0080] The position detection module can also be mounted on a housing, which can be the outer shell of a projection screen, a vehicle interior trim piece, a mounting base for a projection interaction system, or other structures used to fix the position detection module. The housing provides mechanical support for the position detection module and protects the sensors and drive components within it. If the housing has a light-transmitting window, the position detection module can emit detection light and receive reflected light through this window.

[0081] The position detection module can also be mounted on a structure that is relatively fixed to the projection medium. "Relatively fixed" here can mean that the positional relationship between the structure and the projection medium remains essentially unchanged during normal use, or that this positional relationship can be determined through calibration, detection, or parameter configuration. For example, the position detection module can be mounted on a vehicle roof, seat back, interior partition, projection bracket, wall bracket, or display equipment bracket. The detection area of ​​the position detection module can cover the interactive area in front of the projection medium, thus obtaining detection data of the object being detected.

[0082] The position detection module may include a direct time-of-flight sensor. The direct time-of-flight sensor emits detection light and receives reflected light from the object being detected. Based on the time of flight of the detection light from emission to reception, the distance between the object and the sensor can be determined. This distance can be used as part of the detection data for subsequent determination of the object's position.

[0083] The detection light emitted by a DTOF sensor can be infrared light or other suitable light signals for distance measurement. When an object enters the detection range, it reflects the detection light. After receiving the reflected light, the DTOF sensor can calculate the distance value based on the time of flight. This distance value can represent the straight-line distance between the detected object and the DTOF sensor, or it can be processed to represent the distance between the detected object and the projection medium.

[0084] The location detection module acquires detection data, which can include raw detection data or pre-processed data. Raw detection data can be distance values, angle values, and reflection intensity directly output by the DTOF sensor. Pre-processed data can be detection points, candidate object information, or point cloud data generated by the location detection module based on the raw detection data. The system architecture determines whether to receive raw or processed detection data.

[0085] After acquiring the detection data, the position information of the detected object can be determined based on the data. Position information can represent the position of the detected object relative to the position detection module, the position of the detected object relative to the projection medium, or the position of the detected object in the vehicle coordinate system, screen coordinate system, or display screen coordinate system. Position information can include distance, angle, two-dimensional coordinates, three-dimensional coordinates, point cloud center position, object outline position, or other information that can indicate the location of the detected object.

[0086] Optionally, the detection points are converted into coordinate points based on distance and angle values. These coordinate points can represent the position of the detected object or a point on its surface within the detection coordinate system. If multiple coordinate points are obtained in a single scan, a coordinate point cloud is generated based on these points. The coordinate point cloud can include points related to the detected object, as well as background points from the projection medium, points within the vehicle structure, environmental noise points, or other non-target points.

[0087] To determine the location of the detected object from the coordinate point cloud, the coordinate point cloud can be processed. Processing methods may include effective range filtering, background point filtering, noise point filtering, clustering, trajectory tracking, or other point cloud processing methods.

[0088] The location information of the detected object is determined based on the coordinates of the candidate object. For example, the center point of the candidate object point cloud can be used as the location of the detected object. The center point can be the average position of multiple coordinate points, or it can be a position calculated based on the weighted average of reflection intensity. For finger operation scenarios, a point in the point cloud that is closer to the projection medium can also be selected as the location of the detected object, so that the location is closer to the actual projection image position pointed to by the user.

[0089] If the object being detected is a pointing tool, its position can be determined based on the tool's tip. The main body of the pointing tool may form a continuous point cloud. Using the center of the point cloud directly might result in a discrepancy with the user's actual pointing position. Therefore, the tool's tip position can be determined based on the point cloud's shape, direction of extension, or the endpoint closest to the projection medium, and this tip position can be used as the object's position information.

[0090] The location information of the detected object can be either real-time or a smoothed location. Since DTOF ranging and scanning angle may have some errors, the location obtained from a single scan may exhibit slight jitter. Smoothing can be performed on the location of multiple consecutive frames. For example, moving average, low-pass filtering, or amplitude limiting can be used to make the location information more stable, thus reducing cursor jitter or false triggers.

[0091] In a vehicle environment, vibrations during vehicle movement, occupant movement, changes in lighting, or screen shaking can affect detection data. This data is processed by combining temporal continuity, background updates, and distance thresholds. For example, if a detection object appears only in a single scan cycle and subsequently disappears, it can be considered an invalid detection result; conversely, if a detection object appears in multiple consecutive scan cycles and maintains a stable position, it can be identified as a valid detection object.

[0092] When the projection medium is a flexible screen, the screen surface may have slight curvature. The distance between the detected object and the projection medium can be determined based on the background points of the screen. For example, the background distance corresponding to the screen surface can be recorded when there is no user interaction. When the user interacts, the difference between the detected object's position and the corresponding background distance can be used to determine whether the detected object is close to the projection medium. In this way, even if the screen surface is not perfectly flat, it is still possible to determine whether the detected object is within the effective interaction area.

[0093] The detection range can also be set according to the effective display area of ​​the projected image. If the projected image only covers a portion of the projection medium, the position detection module can only detect the area in front of that effective display area. This reduces detection data in irrelevant areas and improves subsequent processing efficiency. If the projected image is large, the scanning angle can be expanded or multiple position detection modules can be set to cover the entire projected image.

[0094] Multiple position detection modules can also be used in combination. For example, in a wide projection screen scenario, position detection modules can be set on the left and right sides of the horizontal support beam. Different position detection modules can cover different areas, or they can detect the same object from different angles. The detection data output by multiple position detection modules can be fused to improve the stability of the detection range and position determination.

[0095] In some implementations, the position detection module can adjust its scanning strategy based on the application scenario. For example, in a playback control interface, the area frequently operated by the user might be concentrated in the playback control bar at the bottom of the display screen; the position detection module can increase the scanning frequency of this area. In a desktop interface or game interface, the user may operate across the entire display screen; the position detection module can adopt a scanning method with more even coverage.

[0096] The detection data from the location detection module can also be used to determine whether the detected object has entered a valid interactive state. For example, when the detected object is far from the projection medium, it can be assumed that the user is merely passing by or pointing, and may not necessarily trigger an interaction; when the detected object enters a set distance range, it can be assumed that the user has an interactive intent; when the detected object stays or moves within the set distance range, interactive actions such as clicking, swiping, or long-pressing can be further determined. This determination can be completed in subsequent steps in conjunction with the mapped coordinate parameters.

[0097] In suspended touch scenarios, there can be a certain gap between the detected object and the projection medium. After the position detection module detects the object, it determines its pointing position in front of the projection medium based on the object's position. The user does not necessarily need to touch the projection medium; they can complete the interaction by approaching, pausing, or moving. This method reduces contact with the projection medium surface and is suitable for flexible screens, vehicle screens, or scenarios where the projection medium needs to be kept clean.

[0098] In contact or near-contact scenarios, the detected object can directly touch the projection medium or be very close to it. The position detection module can determine whether the detected object meets the trigger condition based on changes in distance. For scenarios where users are accustomed to directly clicking the screen, this method can provide a touchscreen-like operating experience; for scenarios where users are not allowed to directly press the screen, a distance threshold can be set so that the trigger is completed when the user approaches the screen.

[0099] In multi-object scenarios, the position detection module can detect multiple objects, which can be multiple fingers of the same user, or fingers or pointing tools of different users. The position detection module can determine multiple positional information based on multiple point cloud data sets, and number and track different detection objects. Subsequently, multi-finger scaling, rotation, dragging, or partitioning operations can be determined based on the positional relationships of the multiple detection objects.

[0100] In rear-seat entertainment systems, passengers on the left and right sides of the second row may simultaneously interact with the projection screen. A position detection module can detect the positions of multiple fingers or pointing tools and distribute interaction events to different display areas based on the mapped positions of these objects on the screen. This allows multiple users to operate the projected image.

[0101] Through the above processing, the position detection module can convert the user's spatial operations in front of the projection medium into detection data, and determine the position information of the detected object based on the detection data.

[0102] Step S303: Determine the mapping coordinate parameters based on the position information. The mapping coordinate parameters include the mapping coordinates of the detected object in the display screen and the motion state information of the detected object.

[0103] Position information is used to represent the position of the detected object in the detection space. This position information can be determined by step S302 and can be represented in the form of two-dimensional coordinates, three-dimensional coordinates, a combination of distance and angle, a set of detection points, or the area where the detected object is located. Mapped coordinates are used to represent the position of the detected object on the display screen, and the motion state information of the detected object is used to represent the current motion state of the detected object. By converting the position information into mapped coordinate parameters, a correspondence can be established between the operations performed by the user in front of the projection medium and the displayed content on the display screen, providing a basis for determining the position and motion state for subsequent interactive operations.

[0104] As one possible implementation, determining the mapping coordinate parameters based on the location information includes: converting the location information of the detected object into mapping coordinate parameters based on a preset coordinate mapping relationship; wherein the preset coordinate mapping relationship is used to characterize the correspondence between the location information of the detected object and the mapping coordinates in the display screen.

[0105] The position detection module and the display screen can use different coordinate systems. The position information output by the position detection module is usually located in the detection coordinate system, which can take the installation position of the position detection module, the position of the detection light emission, or the center of the scanning axis as the origin, and the scanning direction, installation direction, or vehicle direction of the position detection module as the coordinate axis direction.

[0106] For example, when the position detection module includes a DTOF sensor, the detection data can include the distance value of the detected object relative to the DTOF sensor and the scanning angle. Based on this distance value and scanning angle, the coordinates of the detected object in the coordinate system of the DTOF sensor can be determined. When the position detection module can obtain height direction information, the position information can also include the coordinates of the detected object in the height direction.

[0107] The display screen can use a pixel coordinate system. The pixel coordinate system can use the top-left corner of the display screen as the origin, the horizontal direction as the X-axis, and the vertical direction as the Y-axis. Each display position on the screen can be represented by a single pixel coordinate. Alternatively, the display screen can use a normalized coordinate system. In a normalized coordinate system, the horizontal and vertical coordinates can be between 0 and 1. Using normalized coordinates, the mapping process does not directly depend on the specific resolution of the display screen.

[0108] The projection medium can also have a corresponding media coordinate system, which can be established based on the actual size of the projection medium. For example, the upper left corner of the projection medium's display area can be used as the origin, the width of the projection medium as the horizontal axis, and the height of the projection medium as the vertical axis. The position information obtained by the position detection module can be first converted to the media coordinate system, and then converted from the media coordinate system to the pixel coordinate system of the display screen. By setting the media coordinate system, the spatial position of the detected object, the corresponding position on the projection medium, and the operation position on the display screen can be distinguished.

[0109] Preset coordinate mapping relationships can be stored in advance or obtained through calibration after the projection device, projection medium, or position detection module is installed. These relationships can also be stored in the projection interaction device, projection equipment, position detection module, vehicle infotainment system, or cockpit controller. Upon system startup, the corresponding preset coordinate mapping relationship can be read to convert the position information of the detected object into mapped coordinates on the display screen.

[0110] During calibration, the projection device can be controlled to display multiple feature point markers on the projection medium. These feature point markers can be distributed at the edges, corners, and center of the display area. For example, feature point markers can be displayed at the four corners of the display area, or they can be displayed in multiple rows and columns throughout the display area. Feature point markers can be dots, crosshairs, squares, or other graphics that indicate the calibration position.

[0111] Users can use their fingers or pointing tools to point to each feature point in turn. The position detection module detects the objects pointing to the feature point markers, obtains the detection coordinates corresponding to each feature point marker, and associates the detection coordinates with the pixel coordinates of the feature point markers on the display screen, thereby obtaining multiple sets of coordinate correspondences.

[0112] For example, markers that can be recognized by the position detection module can be set at multiple locations on the projection medium, or they can be moved sequentially to set positions using movable calibration components. The position detection module detects the position of each marker or calibration component, and the projection interaction device establishes a coordinate mapping relationship based on the detection results and the coordinates of each set position on the display screen.

[0113] Based on multiple sets of coordinate correspondences, coordinate transformation parameters can be determined. These parameters can describe the translation, scaling, rotation, or projection relationships between the detected coordinates and the displayed coordinates. When the positional relationship between the projection medium and the position detection module is relatively fixed, and the display area of ​​the projection medium is close to a plane, a linear mapping method can be used for coordinate transformation.

[0114] When the position detection module is installed at an angle relative to the projection medium, or when the projection device uses an oblique projection method, a coordinate transformation matrix can be determined based on multiple calibration points, and the position information can be transformed using the coordinate transformation matrix. The coordinate transformation matrix may include one or more of the following: translation parameters, rotation parameters, scaling parameters, and perspective transformation parameters.

[0115] Furthermore, a nonlinear mapping relationship can be established based on multiple calibration points. For example, multiple neighboring calibration points can be selected based on the location of the object being detected, and the interpolation weights can be determined based on the distance between the object and each calibration point. Then, the mapped coordinates can be calculated based on the display screen coordinates corresponding to each calibration point. This method can compensate for local deformation of the projection medium, installation errors, or detection errors.

[0116] The unfolded position of the projection medium may change. For example, the projection screen can be driven by a retracting assembly and can remain at different unfolded heights. When the unfolded height of the screen changes, the position of the display area relative to the position detection module may also change. The projection interaction device can select the corresponding coordinate mapping relationship or adjust the coordinate transformation parameters based on the unfolded length of the screen, the rotation position of the retracting assembly, or the position of the lower edge of the screen.

[0117] When the position or projection direction of the projection device changes, the coordinate mapping relationship can also be redefined. For example, the projection device may have a projection angle adjustment function, or the projection direction may be adjusted according to the position of the vehicle seats. The installation position, projection angle, projection screen size, or keystone correction parameters of the projection device can be obtained, and the mapping coordinates can be corrected based on these parameters.

[0118] When a change in the position of the projection medium, projection device, or position detection module is detected, the projection interaction device can prompt for recalibration, or select another pre-saved set of coordinate mapping relationships based on the current device parameters. By updating the coordinate mapping relationships, the position of the detected object can continue to correspond to its display position on the current display screen.

[0119] The mapping coordinate parameters must include at least the mapping coordinates of the detected object on the display screen. Mapped coordinates can be a single point or a region of coordinates. When the detected object is a user's finger, the coordinates corresponding to the fingertip can be used as the mapping coordinates. When the detected object includes multiple detection points, the detection point corresponding to the fingertip can be determined from among the multiple detection points, or the fingertip position can be determined based on the contour formed by the multiple detection points.

[0120] For example, the detection point closest to the projection medium can be determined as the detection point corresponding to the fingertip. Alternatively, the end position of the object to be detected can be determined based on the extension direction of the set of detection points. When the object to be detected is a stylus or other operating tool, the end of the operating tool closest to the projection medium can be determined as the position for coordinate mapping.

[0121] When the detection results cannot consistently distinguish the fingertip position, the center position can be determined based on the set of detection points corresponding to the detected object, and this center position can be converted into mapped coordinates. The mapped position can also be corrected based on the shape, size, direction of movement, or historical position of the detected object.

[0122] During continuous scanning, the position detection module can obtain the position information of the detected object at multiple times. It then performs coordinate transformation on the position information at each time to obtain a mapped coordinate sequence, which can be used to represent the movement process of the detected object on the display screen.

[0123] The motion state information of the detected object can be determined based on the mapped coordinate sequence, the distance change between the detected object and the projection medium, or a combination of both. The motion state information of the detected object can include at least one of the following: stationary state, moving state, approaching state, moving away state, hovering state, contact state, or leaving state.

[0124] For example, if multiple consecutive mapped coordinates are within a set range and the duration reaches a set time, it can be determined that the detected object is in a stationary state. If the change in distance between multiple consecutive mapped coordinates reaches a set distance, it can be determined that the detected object is in a moving state. If the distance between the detected object and the projection medium continuously decreases, it can be determined that the detected object is in a approaching state; if the distance between the detected object and the projection medium continuously increases, it can be determined that the detected object is in a moving-away state.

[0125] When the distance between the detected object and the projection medium is less than a distance threshold, the detected object can be determined to be in a contact state or a near-contact state. When the detected object enters the interactive area but does not meet the contact state, near-contact state, or other operation triggering conditions, the detected object can be determined to be in a hovering state. When the detected object leaves the detection range, the effective display area, or the set distance range, the detected object can be determined to be in a leaving state.

[0126] The motion state information of the detected object may also include at least one of the following: dwell time, coordinate change trajectory, direction of movement, speed of movement, acceleration of movement, trajectory duration, change in distance between the detected object and the projection medium, or change in relative position between multiple detected objects. This information can be used as part of the mapped coordinate parameters for subsequent determination of the first action and interactive operation.

[0127] The detection data may be affected by vehicle vibration, user hand tremors, ambient light, or measurement errors, causing small variations in the continuously obtained mapped coordinates. To address this, the position information or mapped coordinates can be smoothed. Smoothing can determine the output coordinates based on the current coordinates and multiple historical coordinates, or it can filter out coordinate variations with a magnitude smaller than a set value.

[0128] When the mapped coordinates exceed the valid coordinate range of the display screen, the mapped coordinates can be restricted to the boundary position of the display screen. Alternatively, it can be determined that the detected object is not located within the valid interaction area, or an edge operation area can be set outside the valid display area. When the detected object enters the edge operation area, the corresponding interaction operation can be determined subsequently based on the detected object's motion state information and the mapped coordinate parameters.

[0129] To prevent detected objects from frequently entering and exiting the effective interaction area when moving near its edge, entry and exit thresholds can be set. The entry threshold is used when the detected object enters the effective interaction area, and the exit threshold is used when it leaves. The entry and exit thresholds can be different, thereby reducing frequent switching of movement states at the edge locations.

[0130] In vehicle-mounted scenarios, vibrations generated by vehicle movement can cause relative movement between the position detection module, the projection medium, and the object being detected. The projection interaction device can acquire vehicle vibration information, screen position, or the attitude information of the position detection module, and correct the position information, mapped coordinates, or motion state information of the object being detected based on these information.

[0131] Step S304: Determine the interactive operation based on the mapped coordinate parameters, and control the display screen to respond to the interactive operation.

[0132] Multiple interactive areas can be pre-defined on the display screen. For example, interactive areas can correspond to buttons, icons, list items, input boxes, progress bars, control sliders, or other display controls. After obtaining the mapped coordinates, the projection interactive device can determine whether the mapped coordinates lie within an interactive area. If it does, it can determine a first action based on the mapped coordinate parameters, and then determine whether to perform an interactive operation on the display control corresponding to the interactive area based on the first action.

[0133] The interactive area can be the same as or larger than the display control's display boundary. For example, when the display control is small, an extended area of ​​a certain width can be set around it, and the display control and its extended area together constitute the interactive area. In this way, even if the mapped coordinates deviate from the center position of the display control, the user's operation on the display control can still be identified.

[0134] The interactive area can also be adjusted based on the user's location, detection distance, or display resolution. For example, when the distance between the user and the projection medium is large, the interactive area can be increased; when the controls on the display screen are densely packed, the overlap between adjacent interactive areas can be reduced. When multiple interactive areas overlap, the display control corresponding to the interactive operation can be determined based on the distance between the mapped coordinates and the center position of each display control, the hierarchy of the display controls, or the current operation state of the interface.

[0135] As one possible implementation, determining the interactive operation based on the mapped coordinate parameters includes: determining a first action based on the mapped coordinate parameters, the first action including at least one of a click action, a double-click action, a long-press action, a swipe action, a drag action, and a multi-finger action; and determining the interactive operation based on the first action.

[0136] The first action can be used to characterize the type of action formed by the detected object in front of the projection medium, and the interactive operation can be used to characterize the operation intention or function operation determined based on the first action. The interactive operation can include at least one of the following: hover selection, control selection, page switching, list scrolling, display object movement, display object scaling, display object rotation, playback control, vehicle function control, or split-screen control.

[0137] Different display interfaces can be configured with different correspondences between the first action and the interactive operation, and different judgment conditions can be set for the same coordinate change. For example, the same swipe action can correspond to list scrolling in a list interface, page switching in a multi-page interface, and playback progress adjustment in a video playback interface.

[0138] Click actions can be determined based on mapped coordinates and dwell time. For example, when a detected object enters an interactive area and remains within that area for a predetermined time, the first action can be determined as a click action targeting that interactive area. Click actions can also be determined based on changes in distance between the detected object and the projection medium. For example, if the detected object first approaches the projection medium, then moves away from it after reaching a predetermined distance, and its mapped coordinates remain within the same interactive area throughout this process, the first action can be determined as a click action.

[0139] When determining whether a detected object is moving closer or further away, the judgment can be made based on distance values ​​obtained from multiple detection cycles. A continuously decreasing distance value indicates that the detected object is moving closer to the projection medium, while a continuously increasing distance value indicates that the detected object is moving away from the projection medium. When the detected object moves to within a set distance of the projection medium, the distance condition corresponding to the click action is met.

[0140] When the detected object can come into contact with the projection medium, the mapped coordinates corresponding to the contact position can also be used as the click position. For example, if the distance between the detected object and the projection medium is less than a distance threshold, and this state continues for a set time, the detected object can be considered to be in contact with or close to the projection medium. Thus, the same interaction method can be applied to both contact and hovering operations.

[0141] A double-click action can be determined based on the time interval and positional relationship between two click actions. When two consecutive click actions are detected, the time interval between the two click actions is less than a set interval, and the distance between the mapped coordinates corresponding to the two click actions is less than a set distance, the first action can be determined as a double-click action.

[0142] A long press action can be determined based on the dwell time of the mapped coordinates. When the dwell time of the mapped coordinates within the same interaction area reaches a second preset time, the first action can be determined as a long press action. When determining a long press action, the mapped coordinates do not need to remain unchanged; a certain range of movement is allowed. As long as the mapped coordinates do not exceed this allowed range of movement within the second preset time, the detected object can be considered to remain in the same position.

[0143] The sliding motion can be determined based on the trajectory of coordinate changes. If the mapped coordinates move continuously on the display screen and the trajectory length reaches a set length, the first motion can be determined as a sliding motion. The projection interactive device can determine the sliding direction based on the relative position between the starting and ending points of the trajectory.

[0144] The sliding direction can include left, right, up, down, or other directions. A horizontal slide is defined as the angle between the coordinate change trajectory and the horizontal direction falling within a first angle range; a vertical slide is defined as the angle between the coordinate change trajectory and the vertical direction falling within a second angle range. For curved trajectories, the sliding direction can be determined based on the overall extension direction of the trajectory or the directions of multiple line segments within the trajectory.

[0145] When determining a swipe action, movement speed and operation duration can also be considered. A swipe is defined as a fast swipe when the mapped coordinates move a relatively long distance in a short time; a swipe is defined as a normal swipe when the mapped coordinates move continuously at a lower speed. Fast swipes and normal swipes can correspond to different screen movement distances or page transition speeds.

[0146] The dragging action can be determined based on the selection state and coordinate change trajectory of the detected object. For example, if the detected object begins to move after being clicked, long-pressed, or approached within the interactive area corresponding to a displayed object, that displayed object can be identified as the dragged object, and the first action can be determined as the dragging action. As the mapped coordinates change, the target position corresponding to the dragged object can be determined.

[0147] During the dragging process, the position of the dragged object can be continuously determined based on the current mapped coordinates. After the mapped coordinates enter the target area, if the detected object moves away from the projection medium, leaves the detection range, or remains within the target area for a set time, the dragging can be determined to end, and the corresponding interactive operation can be determined.

[0148] When multiple objects are detected, multi-finger actions can be determined based on the mapped coordinates corresponding to the multiple objects. Each object can be tracked separately, and the distance, relative direction, and positional changes between the mapped coordinates can be determined.

[0149] Multi-finger gestures can include zooming, rotating, and swiping. When the distance between two mapped coordinates increases, the multi-finger gesture is considered a zoom-in gesture; when the distance decreases, it is considered a zoom-out gesture. When performing a zoom-in or zoom-out gesture, the midpoint between the two mapped coordinates can be used as the zoom center, and the zoom ratio is determined based on the change in distance.

[0150] When the direction of the line connecting two mapped coordinates changes, the multi-finger action can be identified as a rotational action. The projection interaction device can determine the rotation angle based on the angle of change in the line direction. When multiple detected objects move along the same direction, the multi-finger action can be identified as a multi-finger swipe action.

[0151] Once the first action is determined, the corresponding mapped coordinates, interactive area, currently displayed interface, and currently running application can be used to determine the interactive operation. For example, click, double-tap, or long-press actions can correspond to control selection, control confirmation, hover selection, or page switching; swipe actions can correspond to list scrolling, page switching, or playback control; drag actions can correspond to moving the displayed object or controlling the slider; and multi-finger actions can correspond to scaling the displayed object, rotating the displayed object, switching display modes, or split-screen control.

[0152] Hover selection can be determined based on the mapped coordinates and the distance between the detected object and the projection medium. When the detected object enters an interactive area but has not yet met the criteria for a click action, it can be determined that the detected object is in a hover state, and the interactive operation is identified as hover selection. Hover prompts and operation confirmations can be set separately. For example, after the mapped coordinates enter the interactive area, hover selection can be determined first; after the detected object continues to move closer to the projection medium or remains there for a set time, the click action and corresponding interactive operation can then be determined.

[0153] To reduce erroneous operations, corresponding judgment conditions can be set for different first actions. Judgment conditions can include time conditions, distance conditions, trajectory conditions, speed conditions, and position conditions. The first action is only determined when the corresponding judgment condition is met. For example, if the mapped coordinates stay in the interactive area for too short a time and do not form a coordinate change trajectory that satisfies the sliding conditions, the first action may not be determined. Similarly, if the detected object quickly passes in front of the display screen but does not stay in any interactive area, the interaction operation may also not be determined.

[0154] It can also exclude non-operational objects based on the size, shape, or movement of the detected object. For example, when the position detection module detects a large object passing through the detection range, it can determine that the object does not belong to a finger or operating tool used for interaction, and thus does not determine the first action and interaction operation based on the object's mapped coordinates.

[0155] For consecutively identified identical first actions, an operation interval can be set. After a click action is confirmed, repeated clicks within the same interactive area can be ignored for a set period. Recognition of the next action can resume once the detected object leaves the interactive area or the operation initiation conditions are met again.

[0156] When the detected object is at the edge of the interactive area, a state-preserving approach can be used. For example, once it is determined that the detected object has entered the first interactive area, the selection state of the first interactive area is only deselected when the mapped coordinates exceed a certain distance beyond the first interactive area. This method can avoid frequently switching the selected object when the mapped coordinates change near the boundary of adjacent interactive areas.

[0157] Once the interactive operation is determined, a corresponding interactive instruction can be generated. The interactive instruction can include one or more of the following: operation type, mapped coordinates, operation object identifier, trajectory information, operation duration, and operation parameters. For example, a click instruction can include the click location and the identifier of the clicked control; a swipe instruction can include the trajectory start point, trajectory end point, and swipe direction; a drag instruction can include the dragged object, current position, and target position; a zoom instruction can include the zoom center and zoom ratio; and a rotate instruction can include the rotation center and rotation angle.

[0158] Interactive commands can be sent to the vehicle's infotainment system, cockpit controller, projection device, or other electronic devices running display applications. The device receiving the interactive command can perform corresponding interface processing based on the current display interface, the type of interactive command, and the mapped coordinates.

[0159] As one possible implementation, controlling the display screen to respond to interactive operations specifically includes: controlling the display screen to execute a first response operation based on a first action. The first response operation includes at least one response operation type selected from interface adjustment, target selection, playback adjustment, volume adjustment, and brightness adjustment.

[0160] The first response operation can be the display processing result determined based on the first action and the interaction operation. When the first action is a click, double-click, or long-press action, the first response operation can include target selection, interface adjustment, or playback adjustment. When the first action is a swipe or drag action, the first response operation can include interface adjustment, playback adjustment, volume adjustment, or brightness adjustment. When the first action is a multi-finger action, the first response operation can include interface adjustment, such as scaling, rotating, or adjusting the split-screen area of ​​the displayed object.

[0161] When the control display screen performs the first response operation, the vehicle's infotainment system, cockpit controller, or application can generate updated display data, which is then output to the projection device. The projection device projects the corresponding display image onto the projection medium based on the updated display data. Alternatively, the projection device can switch the image frames to be projected to update the display image.

[0162] When the first response operation is interface adjustment, it can include changing control states, switching pages, scrolling lists, moving displayed objects, adjusting the size or orientation of displayed objects, updating text content, playing animations, or changing video playback status. When the first response operation is target selection, it can set the displayed controls, list items, icons, files, or other displayed objects corresponding to the mapped coordinates to the selected state. When the first response operation is playback adjustment, it can control video, audio, or other media content to play, pause, fast forward, rewind, or adjust the playback progress. When the first response operation is volume adjustment, it can adjust the media playback volume or the vehicle's audio system volume. When the first response operation is brightness adjustment, it can adjust the display brightness of the screen, the projection brightness of the projection device, or the interior lighting brightness of the vehicle.

[0163] When the first action is a click, the display screen can be controlled to select a target or adjust the interface based on the corresponding interactive operation. If the interactive operation is a control selection, the display control corresponding to the mapped coordinates can be set to the selected state; if the interactive operation is a page switching, the display screen can be controlled to switch to the corresponding page; if the interactive operation is a switch control, the state of the switch function can be changed; if the interactive operation is an application launch, the corresponding application can be launched and the application interface can be displayed.

[0164] When the first action is a long press, the display screen can be controlled to show a shortcut menu, deletion prompt, editing interface, or other operation content corresponding to the currently displayed object, based on the interaction operation corresponding to the long press action. For example, when the user performs a long press action on the position corresponding to the application icon, the application management menu can be displayed; when the user performs a long press action on the position corresponding to the text or image, the user can enter the selection or editing state.

[0165] When the first action is a swipe, the screen can be adjusted based on the swipe direction, distance, and speed, including interface adjustments, playback adjustments, volume adjustments, or brightness adjustments. For example, when the screen includes a list, a vertical swipe can scroll the list up or down; when the screen includes multiple pages, a horizontal swipe can switch pages; when the screen is a video playback interface, a horizontal swipe can adjust the playback progress, and a vertical swipe can adjust the volume or display brightness.

[0166] When the first action is dragging, the user can control the display screen to perform interface adjustments, volume adjustments, or brightness adjustments. For example, the position of the dragged object can be continuously updated based on the mapped coordinates, and the movement process of the dragged object can be displayed on the screen; the position of the control slider can also be adjusted based on the mapped coordinates, and the air conditioning temperature, seat position, volume, or interior lighting brightness can be adjusted accordingly.

[0167] When the first action is a multi-finger gesture, the user can control the display screen to perform interface adjustments. For example, the scaling ratio can be determined based on the distance change between multiple mapped coordinates, and the displayed objects can be updated according to the scaling ratio; the rotation angle can be determined based on the relative angle change between multiple mapped coordinates, and the displayed objects can be updated according to the rotation angle; the split-screen area can also be adjusted, the display mode can be switched, or the function menu can be opened based on the multi-finger swipe gesture.

[0168] See Figure 7 The diagram shown illustrates a detection method involving multiple objects. Figure 7 In the scenario shown, the position detection module can simultaneously detect two objects and determine their corresponding mapped coordinates. When the distance between the two mapped coordinates continuously increases, the first action is determined to be a zoom-in action; when the distance between the two mapped coordinates continuously decreases, the first action is determined to be a zoom-out action. The zoom center can be determined based on the midpoint between the two mapped coordinates, and the zoom ratio can be determined based on the distance change to zoom in or out on images, videos, maps, or other display objects on the screen.

[0169] See Figure 8 The diagram shown illustrates another detection method involving multiple objects. Figure 8In the scenario shown, the position detection module can simultaneously detect three objects and determine the corresponding mapped coordinates for each. When the three objects enter the effective area of ​​the display screen within a set time and meet preset dwell time, positional relationship, or movement conditions, the first action can be determined as a three-finger screenshot action. After determining the three-finger screenshot action, the image data of the current display screen can be acquired, and the corresponding screenshot image can be generated.

[0170] The first response action can also include feedback. Feedback is used to indicate that the interactive action has been recognized or executed. Feedback can include changes in control color, changes in control size, border display, cursor movement, progress indicators, text prompts, or animation effects.

[0171] When executing the first response operation based on the interactive operation control screen, the update can be performed once after the interactive operation is confirmed, or it can be continuously updated during the execution of the interactive operation. For example, a click action can update the display screen once after the click condition is met; a swipe action, drag action, or multi-finger action can continuously update the display screen as the mapped coordinate parameters change.

[0172] When continuously updating the display screen, display data can be generated according to a set update cycle. The display update cycle can be the same as the scanning cycle of the position detection module or the display refresh cycle of the projection device. When the generation frequency of position detection data differs from the display refresh frequency, the mapped coordinates can be interpolated, or the mapped coordinates corresponding to the current display time can be selected.

[0173] To reduce the latency between user actions and screen response, the first action can be determined after obtaining the current mapped coordinate parameters. For continuous actions such as swiping, dragging, or multi-finger gestures, the first response operation can be executed on the display screen based on the obtained partial trajectory before the complete trajectory is formed, and the operation parameters can be continuously corrected during subsequent detection.

[0174] If detection data is temporarily lost during an interactive operation, the handling method can be determined based on the duration of the loss. If the detection data is recovered within a short time, the coordinate change trajectory can be further determined by combining the mapped coordinates before and after the loss; if the detection data loss time exceeds a set time, the current interactive operation can be terminated. For continuous operations such as dragging or scaling, after detection data loss, the displayed object can be kept in the last determined position, or the unfinished operation can be undone.

[0175] After controlling the projection device to update the display screen, the results of the interactive operation can also be recorded. The results can include whether the operation was successful, whether the display screen has been updated, or whether the application has completed the corresponding processing. If the operation fails, the original display screen can be maintained, or a prompt can be displayed to inform the user to try again.

[0176] In in-vehicle applications, the displayed screen can be a video playback interface, entertainment application interface, vehicle control interface, seat control interface, air conditioning control interface, navigation interface, or communication interface. Different interfaces can be configured with the allowed first action, interactive operation, and first response operation according to functional requirements.

[0177] For example, when the vehicle is in motion, some interactive functions can be restricted based on the vehicle's status. The cockpit controller can obtain the vehicle's speed, gear position, or driving status, and determine the permitted interactive operations based on this status. For operations that may affect driving, the corresponding interactive commands may not be executed, or only the position detection module corresponding to the area where the rear passengers are located may be allowed to trigger the operation.

[0178] When the projection medium is positioned behind the first-row seats, second-row passengers can move their fingers in front of the medium to click, swipe, or drag the projected image. When the projection medium is positioned on the roof, passengers can control the projected image by raising their hands. The position detection module can determine the operating area based on the installation location and detection range, and map the position of the detected object within the operating area onto the display screen.

[0179] When the display screen comprises multiple split-screen areas, the corresponding interactive operation can be determined and sent to the corresponding application based on the split-screen area to which the mapped coordinates belong. For example, the first split-screen area of ​​the display screen is used to play video, and the second split-screen area is used to display the vehicle control interface. When the mapped coordinates are detected to be located in the first split-screen area, video playback can be controlled; when the mapped coordinates are detected to be located in the second split-screen area, vehicle functions can be controlled.

[0180] When multiple users operate simultaneously, corresponding operation permissions can be determined based on the location of different detected objects, their seating area, or object identifiers. When the mapped coordinates of different users are located in different display areas, each user's interaction can be responded to separately. When multiple users operate the same display control simultaneously, the interaction to be executed can be determined based on the operation time, user permissions, or preset priority.

[0181] By determining the first action based on mapped coordinate parameters, determining the interactive operation based on the first action, and then controlling the display screen to execute the first response operation based on the first action, users can interact with the displayed content through a detection object located in front of the projection medium. The detection object can be in contact with the projection medium or maintain a certain distance from it, thus adapting to both contact interaction and hovering interaction.

[0182] As one possible implementation method, determining the location information of the detection object specifically includes: determining the distance between the detection object and the location detection module; and determining the location information of the detection object based on the distance between the detection object and the location detection module.

[0183] After detecting the object, the position detection module obtains the distance value between the object and the module, and determines the object's position information based on this distance value. The position information may include at least one of the following: the distance between the object and the module, the distance range of the object, and the object's position relative to the projection medium.

[0184] When the position detection module obtains multiple distance values, it can determine the distance value used to represent the position of the detected object from among these values. For example, the minimum value among the multiple distance values ​​can be used as the distance between the detected object and the position detection module, so that the position on the detected object that is closer to the position detection module is taken as the position of the detected object.

[0185] Alternatively, multiple distance values ​​belonging to the same detection object can be averaged, and the averaged distance value can be used as the distance between the detection object and the position detection module. If there is a distance value that differs significantly from the other distance values, that distance value can be considered an outlier and not used to determine the position of the detection object.

[0186] The positional relationship between the position detection module and the projection medium can be predetermined. Given the distance between the position detection module and the projection medium, the position of the object being detected relative to the projection medium can be determined based on the distance between the object being detected and the position detection module.

[0187] For example, a reference distance between the position detection module and the projection medium can be obtained in advance. When the object being detected enters in front of the projection medium, a difference arises between the distance value detected by the position detection module and the reference distance. Based on the difference between the distance value corresponding to the object being detected and the reference distance, the interval between the object being detected and the projection medium can be determined.

[0188] The distance between the detected object and the projection medium can be used to determine whether the detected object has entered the interactive area. For example, if the distance between the detected object and the projection medium is greater than a first distance threshold, it can be considered that the detected object has not yet entered the interactive area; if the distance is less than or equal to the first distance threshold, it can be considered that the detected object has entered the interactive area.

[0189] The position detection module can pre-obtain the background distance corresponding to a fixed object within the detection range. When the difference between the currently detected distance value and the background distance reaches a set value, it can be determined that the object has entered the detection range, and the position of the object can be determined based on the current distance value.

[0190] For example, once the projection screen is fully extended, the distance between the position detection module and the projection screen can be used as the background distance. When the user's finger enters in front of the projection screen, the currently detected distance value can be less than the background distance. Based on the difference between the current distance value and the background distance, the interval between the user's finger and the projection screen can be determined.

[0191] Background distance can be obtained when the projection screen is fully extended and there are no objects to be detected within the detection range. When the extended position of the projection screen changes, the background distance can be re-acquired or adjusted according to the extended position of the projection screen.

[0192] The position detection module can continuously obtain the distance value corresponding to the detected object according to a preset period. Based on the changes in multiple consecutive distance values, the position change of the detected object can be determined. For example, when the distance value continuously decreases, it can be considered that the detected object is moving closer to the position detection module; when the distance value continuously increases, it can be considered that the detected object is moving away from the position detection module; when the distance value remains within a set range, it can be considered that the position of the detected object remains basically unchanged.

[0193] The motion state of the object relative to the projection medium can also be determined by the change in the interval between the object being detected and the projection medium. When the interval continuously decreases, it can be considered that the object is approaching the projection medium; when the interval continuously increases, it can be considered that the object is moving away from the projection medium; when the interval remains within a set range for a preset time, it can be considered that the object is in a stationary state.

[0194] Distance detection results may be affected by hand tremors, vehicle vibrations, or detection errors. Multiple consecutively acquired distance values ​​can be smoothed. For example, the distance value representing the detected object's position can be determined based on the current distance value and several historical distance values ​​to reduce position state switching caused by small variations in distance values.

[0195] Therefore, by determining the distance between the object being detected and the position detection module, the projection interaction device can determine the position of the object being detected relative to the position detection module or the projection medium, and can determine the object's approaching, moving away, or remaining state based on changes in distance.

[0196] As one possible implementation, the position detection module further includes a driving module, which controls the position detection module to scan within the detection range; and determines the position information of the detected object based on the distance between the detected object and the position detection module, specifically including: determining the position information of the detected object based on the distance and the scanning angle of the position detection module.

[0197] While distance alone allows us to determine the proximity between the detected object and the position detection module, it's difficult to directly pinpoint the specific area in front of the projection medium where the object is located. Therefore, the driving module can change the detection direction of the position detection module, enabling it to sequentially detect objects in different directions within its detection range. This way, while obtaining the distance, we can also determine the corresponding detection direction.

[0198] The drive module can be connected to the position detection module to drive the position detection module to rotate, swing, or change its detection direction according to a preset trajectory. Alternatively, the drive module can also achieve scanning by changing the direction of light propagation without moving the position detection module as a whole. For example, the drive module may include a motor, a rotating shaft, a swing mechanism, or other structures capable of changing the detection direction. Different structures can be selected based on installation space, scanning range, response speed, and cost requirements.

[0199] When the position detection module scans under the action of the drive module, it can use a certain direction as a reference direction and record the angle between the current position detection direction and that reference direction. This angle can be used as the scanning angle. The reference direction can be determined according to the installation position of the position detection module. For example, it can be the direction pointing to the center area of ​​the projection medium, or it can be the horizontal or vertical edge of the projection medium.

[0200] When the position detection module detects an object at a certain scanning angle, it can obtain the distance corresponding to that scanning angle. This distance represents the distance between the detected object and the position detection module, while the scanning angle represents the position of the detected object relative to the direction in which the position detection module is located. Therefore, the distance and scanning angle can be combined to form the detection result, used to determine the position of the detected object within the detection range.

[0201] See Figure 9 As shown, Figure 9 This is a schematic diagram illustrating how to determine the location information of a detected object based on distance and scanning angle. Figure 9In this system, the position detection module can serve as the origin of the detection coordinate system. The distance between the object being detected and the position detection module is d, and the scanning angle of the current detection direction of the position detection module relative to the reference direction is θ. Based on the distance d and the scanning angle θ, the lateral and longitudinal positions of the object being detected in the detection coordinate system can be determined.

[0202] After determining the position of the object to be detected in the detection coordinate system, the position of the object relative to the projection medium can be determined by combining the installation relationship between the position detection module and the projection medium.

[0203] In one implementation, distance and scanning angle can be converted into coordinate values. If a detection coordinate system is established with the position detection module as the origin, the horizontal and vertical positions of the detected object in the detection coordinate system can be determined based on the distance d and the scanning angle θ.

[0204] The relative position between the position detection module and the projection medium can be pre-calibrated. The calibration can include the installation height, installation offset, installation angle of the position detection module relative to the projection medium, and the distance between the position detection module and the projection medium. Based on these calibration parameters, the position in the detection coordinate system can be converted to the position in the projection medium coordinate system.

[0205] The coordinate system of the projection medium can correspond to the display area of ​​the projection medium. For example, a corner point of the projection medium can be used as the origin, the horizontal direction of the projection medium as one coordinate axis, and the vertical direction of the projection medium as another coordinate axis. After the position detection module obtains the position of the detected object in the projection medium coordinate system, it can subsequently determine the mapped coordinates on the display screen based on this position.

[0206] Scanning can be performed according to a preset scan cycle, which can be determined based on interaction response requirements and processing capabilities. For scenarios with high interaction response requirements, such as clicks, swipes, and drags, a shorter scan cycle can be used to promptly obtain changes in the position of the detected object. For scenarios that only require determining proximity or dwell time, a longer scan cycle can be used to reduce processing load and power consumption.

[0207] Each scan can include multiple scanning angles. The position detection module can perform one or more distance measurements at each scanning angle and store the obtained distances in association with the corresponding scanning angles. If no valid reflection result is detected at a certain scanning angle, the detection result corresponding to that scanning angle can be marked as invalid. If a valid distance is detected at a certain scanning angle, that distance and that scanning angle can be used as a valid detection point.

[0208] Valid detection points can include at least one of the following: distance value, scanning angle, reflection intensity, and acquisition time. Distance value and scanning angle are used to determine location, reflection intensity can be used to determine the reliability of the detection result, and acquisition time can be used to distinguish detection results within different scanning cycles. For multiple valid detection points obtained within the same scanning cycle, the location of the object to be detected can be determined based on their distance, angle, and continuity.

[0209] When the object being detected occupies multiple adjacent scanning angles, the position detection module can obtain multiple valid detection points. Based on these valid detection points, a representative position of the object can be determined. For example, a detection point with higher reflectivity can be selected as the representative detection point, or a detection point with a smaller distance can be selected. Alternatively, the positions of multiple valid detection points can be averaged to obtain the position of the object being detected.

[0210] If the distance or angle differences between multiple valid detection points are significant, they can be grouped first. Detection points belonging to the same detection object can be grouped together, and different groups of detection points can correspond to different objects or different parts. For each group of detection points, their corresponding positions can be determined separately. In this way, when there are multiple detection objects within the detection range, the position information of each detection object can be determined separately.

[0211] The position detection module can define the effective detection area based on the scanning angle range. For example, it can use only the detection results within the corresponding angle range in front of the projection medium, excluding those outside this range. When the position detection module is mounted on the transverse support beam or winding assembly of the projection medium, the scanning range can cover at least a portion of the area in front of the projection medium. The drive module can control the position detection module to scan from one side to the other, or to scan back and forth around the center area of ​​the projection medium. The scanning range can be 180°, or it can be set to a range less than or greater than 180° depending on the actual installation position.

[0212] The scanning angle of the drive module can be obtained through an encoder, Hall sensor, stepper motor step count, drive current control information, or other angle feedback information. If a stepper motor is used, the scanning angle can be determined based on the stepper motor step count. If a motor with angle feedback is used, the scanning angle can be determined based on the angle feedback signal.

[0213] To improve the stability of the position determination results, the scanning angle can be calibrated. After the position detection module is installed, the zero point and range of the scanning angle can be determined. If the drive module develops an angular offset after long-term use, the scanning angle can be updated by recalibrating or resetting the zero point, thus reducing the impact of angular errors on the position determination results.

[0214] Once the distance and scanning angle are determined, the position information of the object to be detected can be determined according to a preset conversion relationship. This preset conversion relationship can be calculated using installation parameters or obtained through calibration. When using the calibration method, calibration points can be set at multiple known locations on the projection medium. The distance and scanning angle corresponding to each calibration point detected by the position detection module are recorded, and a correspondence between the detection results and the position of the projection medium is established.

[0215] The number of calibration points can be determined based on the size of the projection medium and the required positional accuracy. More calibration points result in a conversion relationship closer to the actual installation state; fewer calibration points simplify the calibration process. For a flat projection screen, fewer calibration points can be used to establish a planar mapping relationship. If the projection medium is curved, tilted, or has installation errors, more calibration points can be added to improve the accuracy of the positional conversion results.

[0216] In subsequent use, when the position detection module obtains new distances and scanning angles, it can determine the position of the detected object relative to the projection medium based on this preset conversion relationship. This position can be represented as coordinates on the projection medium or as coordinates in the space in front of the projection medium. If subsequent interactions only require determining whether the detected object corresponds to a certain display area, this position can also be converted into an area number or control range.

[0217] Distance and scanning angle can also be used to determine changes in the position of the detected object. After obtaining the position of the detected object within multiple consecutive scanning cycles, the position detection module can determine whether the detected object has moved based on the position changes. If the position changes continuously laterally along the projection medium, it can be used to determine lateral sliding; if the position changes continuously vertically along the projection medium, it can be used to determine longitudinal sliding; if the position remains within a small range, it can be used to determine stationary movement.

[0218] To avoid positional jumps caused by scanning jitter, the positions obtained from consecutive scanning cycles can be smoothed. Smoothing can employ moving averages, weighted averages, or amplitude limiting. If the position obtained in a particular scan differs significantly from the positions in preceding and subsequent scan cycles, and this difference does not conform to the continuous movement trend of the object being detected, that position can be treated as an anomaly.

[0219] When the object being detected moves rapidly, detection points obtained at different scanning angles may correspond to different positions of the object at different times. The projection interaction device can process the detection points based on the acquisition time. For example, it can determine the positional change of the object within a scanning cycle based on the acquisition time of the detection points, or use detection points within the same time range to determine a single positional result. This reduces the impact of time differences during the scanning process on the positional determination result.

[0220] The drive module can scan at a fixed angular velocity or a non-fixed angular velocity. With a fixed angular velocity, the relationship between the scanning angle and time is relatively easy to determine; with a non-fixed angular velocity, the scanning density can be increased in areas with frequent interactions and decreased in areas with fewer interactions. For example, a higher scanning density can be set in the central region of the projection medium to improve the position recognition accuracy of frequently used operating areas.

[0221] The drive module can also adjust the scanning method based on the detection results. If a detection object is continuously detected within a certain angular range, the scanning frequency can be increased or the scanning range can be reduced within that range to track the detection object. If no detection object is detected within a preset time, the scanning frequency can be decreased or the scanning range can be expanded for a re-search. In this way, both response speed and power consumption can be balanced.

[0222] As one possible implementation method, determining the location information of the detection object specifically includes: generating a coordinate point cloud based on the detection points corresponding to the detection object; and filtering the coordinate point cloud to determine the location information of the detection object.

[0223] After the location detection module obtains multiple detection points, these points can be converted into coordinate points, and a coordinate point cloud can be generated based on these coordinate points. The coordinate point cloud can represent a set of spatial positions corresponding to the detected object within the detection range. Compared with a single detection point, the coordinate point cloud can reflect the range, boundaries, and positional distribution of the detected object, thus improving the stability of location information determination.

[0224] Each detection point can be converted into a coordinate point based on its corresponding detection data. The coordinate point can be located in the detection coordinate system or the projection medium coordinate system. If the coordinate point cloud is generated in the detection coordinate system first, it can be transformed to the projection medium coordinate system in subsequent processing according to a preset coordinate transformation relationship. If the coordinate transformation has already been completed, the coordinate point cloud can also be generated directly in the projection medium coordinate system.

[0225] A coordinate point cloud can include coordinate points obtained within a single scan cycle or within multiple consecutive scan cycles. Using coordinate points from a single scan cycle results in a faster position response; using coordinate points from multiple consecutive scan cycles improves the stability of position determination and reduces the impact of single-detection errors on the results.

[0226] After generating the coordinate point cloud, it can be filtered. As one possible implementation, the coordinate point cloud can be filtered, specifically including: processing the coordinate point cloud to remove at least one of the following: background points of the projection medium, invalid detection points, and noise points. For example, coordinate points that are outside the detection range can be removed, as can coordinate points that are not in the interaction area in front of the projection medium. Coordinate points with obviously abnormal distance, coordinates, or reflection intensity can be excluded from subsequent position determination.

[0227] Filtering can also include distance filtering. For coordinate points corresponding to the same object being detected, valid points can be selected based on a distance range. For example, coordinate points within a set distance range can be retained, while those that are too close or too far away can be removed. This avoids misusing detection points corresponding to projection media, vehicle interior structures, or other irrelevant objects for determining the location of the object being detected.

[0228] Filtering can also include area filtering. The effective interaction area in front of the projection medium can be predetermined, and only coordinate points falling within this effective interaction area can be retained. The effective interaction area can be determined based on the projection medium size, the installation location of the position detection module, and the user's operable range. Coordinate points falling outside the effective interaction area can be treated as invalid points.

[0229] For example, you can select the point cloud cluster closest to the projection medium, the point cloud cluster with the most points, or the point cloud cluster located within the current interaction area. If multiple point cloud clusters meet the conditions, you can combine the position information from the previous moment to determine the point cloud cluster with higher continuity to the current interaction action.

[0230] After filtering the coordinate point cloud, the location information of the detected object can be determined based on the filtered coordinate points. Determination methods can include taking the center point, boundary points, the nearest point, or a weighted center point. Different methods can be selected based on the interaction requirements.

[0231] For example, when it's necessary to determine the location of a user's finger, the coordinates of the point cloud closest to the projection medium can be used as the location of the object to be detected. This makes the location information closer to the user's actual position. When the overall location of the object to be detected needs to be obtained, the location of the object can be determined based on the center point of the filtered point cloud.

[0232] When the coordinate point cloud becomes discontinuous within a short period of time, compensation can be made by combining historical positions. For example, if the number of valid coordinate points in a certain period is small, but the point cloud with continuous positions is detected in the preceding and following periods, the position of the current period can be estimated based on the positions of the preceding and following periods. This can reduce position interruptions caused by short-term occlusion or missed detection points.

[0233] By generating a coordinate point cloud based on the detection points and filtering the point cloud, relatively stable object location information is obtained even when the detection results contain noise, missed points, or outliers. This location information can be used for subsequent coordinate mapping and interactive operation recognition.

[0234] As one possible implementation, the method further includes: displaying a cursor on a display screen based on mapped coordinate parameters, wherein the cursor is used to indicate the current position of the detected object on the display screen.

[0235] See Figure 10 As shown, Figure 10 This diagram illustrates the correspondence between the detected object and the displayed cursor position. After determining the mapping coordinate parameters, the cursor's display position on the screen can be determined based on these coordinates. The cursor's horizontal position corresponds to the horizontal coordinate of the mapping coordinates, and the cursor's vertical position corresponds to the vertical coordinate of the mapping coordinates. When the display screen updates, the cursor can be drawn at the corresponding display position and projected onto the projection medium by a projection device.

[0236] When the position of the detected object changes, the mapping coordinate parameters can be updated accordingly. The cursor position can be updated based on the updated mapping coordinate parameters, allowing the cursor to follow the detected object on the display screen. For continuously moving detected objects, the cursor position can be updated according to the refresh cycle of the display screen or the update cycle of the mapping coordinate parameters.

[0237] The cursor can be an arrow, dot, circle, crosshair, hand, or other graphic used to indicate position. The cursor's color, size, transparency, or display style can be set according to the background color of the displayed screen, the current interactive area, or the detection state to distinguish the cursor from other content on the displayed screen.

[0238] When the mapped coordinates are within the interactive area corresponding to the display control, the cursor's display state can be changed. For example, the cursor's color, size, or graphic shape can be changed to indicate that the current cursor is in an operable area. After the mapped coordinates leave the interactive area, the cursor's initial display state can be restored.

[0239] When the object to be detected is not yet detected, the mapped coordinates are invalid, or the mapped coordinates are outside the effective display area of ​​the screen, the cursor can be hidden, or the cursor can be kept at the position corresponding to the last valid mapped coordinate. The holding time can be set according to the duration of the detection data loss. After the set time has elapsed, the cursor can be hidden to prevent it from continuously remaining at a position that does not correspond to the actual position of the object to be detected.

[0240] By displaying a cursor on the screen, a visual cue can be provided to the user to indicate the current position of the detected object, enabling the user to confirm the correspondence between the detected object and its position on the screen.

[0241] Based on the projection interaction method described above, this application can also provide a projection interaction device that can be used to execute the above projection interaction method. The relevant features can be found in the above method embodiments, and will not be repeated here.

[0242] Figure 11 A possible structural schematic diagram of a projection interaction device provided in this application is shown. The projection interaction device may include a projection module, a position determination module, a coordinate determination module, and an interaction module. The projection module controls the projection device to project the display screen onto the projection medium; the position determination module detects a detection object within the detection range using a position detection module and determines the position information of the detection object; the coordinate determination module determines mapping coordinate parameters based on the position information, including the mapping coordinates of the detection object in the display screen and the motion state information of the detection object, which represents the motion state of the detection object; the interaction module determines the interaction operation based on the mapping coordinate parameters and controls the display screen to respond to the interaction operation.

[0243] For example, taking the functional steps of the above projection interaction method as an example, the projection module can be used to control the projection device to receive display data and project the display image corresponding to the display data onto the projection medium. The projection module can also be used to control the projection device to perform image scaling, image offset, keystone correction, geometric correction, brightness adjustment, focus adjustment, or other projection parameter adjustments.

[0244] The position determination module can be used to acquire the detection data output by the position detection module and determine the position information of the detected object based on the detection data. Detection data may include distance values, scanning angles, reflection intensity, detection points, coordinate point clouds, or other data related to the position of the detected object. Position information may include the position of the detected object relative to the position detection module, the position of the detected object relative to the projection medium, the position of the detected object in the projection medium coordinate system, or other data that can characterize the position of the detected object.

[0245] In one implementation, the position determination module can be used to determine the distance between the detected object and the projection medium, and to determine whether the distance is within a set distance range. If the distance between the detected object and the projection medium is within the set distance range, the position determination module can determine the position information of the detected object. If the distance between the detected object and the projection medium is not within the set distance range, the position determination module may not output the corresponding position information, or may mark the corresponding position information as invalid.

[0246] In one implementation, the location determination module can generate a coordinate point cloud based on the detection points corresponding to the detection object, and then filter the coordinate point cloud. The filtering process may include removing at least one of the following: background points of the projection medium, invalid detection points, and noise points. The location determination module can then determine the location information of the detection object based on the filtered coordinate point cloud.

[0247] In one implementation, the position detection module may further include a driving module, and the position determination module may obtain the scanning angle corresponding to the scanning of the position detection module within the detection range, and determine the position information of the detection object based on the distance between the detection object and the position detection module and the scanning angle.

[0248] The coordinate determination module can be used to determine the mapped coordinate parameters based on the position information. The mapped coordinates in the mapped coordinate parameters can represent the corresponding position of the detected object on the display screen. The coordinate determination module can convert the position information of the detected object in the detection coordinate system or the projection medium coordinate system into mapped coordinates on the display screen according to the preset coordinate mapping relationship.

[0249] The preset coordinate mapping relationship can be used to characterize the correspondence between the position information of the detected object and the mapped coordinates on the display screen. The preset coordinate mapping relationship can be obtained through calibration or pre-stored after the projection device, projection medium, and position detection module are installed. When the unfolded position of the projection medium, the projection position of the projection device, the projection area, or the effective display area of ​​the display screen changes, the coordinate determination module can update the preset coordinate mapping relationship.

[0250] The motion state information of the detected object can be determined by the coordinate determination module based on the mapped coordinates at multiple consecutive moments. The motion state information of the detected object can include at least one of the following states: the detected object is in a stationary state, a moving state, a nearing state, a faring state, a contact state, a hovering state, or a state that has left the detection range.

[0251] For example, the coordinate determination module can determine whether the detected object is in a moving state based on the changing distance, direction, speed, or time between multiple consecutive mapped coordinates. If the mapped coordinates remain within a set range for a set time, the detected object can be determined to be in a stationary state. If the distance between the detected object and the projection medium continuously decreases, the detected object can be determined to be approaching; if the distance between the detected object and the projection medium continuously increases, the detected object can be determined to be moving away.

[0252] The interaction module can determine the click action based on the mapped coordinates, dwell time, and the motion state information of the detected object. When the mapped coordinates enter the interaction area and remain within the interaction area for a first set time, the first action can be determined as a click action. When the detected object first approaches the projection medium, then moves away from the projection medium after reaching a set distance, and the mapped coordinates are within the same interaction area during this process, the first action can also be determined as a click action.

[0253] The interaction module can also determine double-click actions based on the time interval and positional relationship between two clicks. It can determine long-press actions based on the dwell time of mapped coordinates within the interaction area. Furthermore, it can determine swipe or drag actions based on coordinate change trajectories, movement direction, movement speed, and the detected object's motion state.

[0254] When the interaction module detects multiple objects, it can determine the multi-finger action based on the mapped coordinates corresponding to the multiple objects. For example, if the distance between two mapped coordinates increases, the multi-finger action can be determined as a zoom-in action; if the distance between two mapped coordinates decreases, the multi-finger action can be determined as a zoom-out action; if the direction of the line connecting two mapped coordinates changes, the multi-finger action can be determined as a rotation action; and if multiple objects move in the same direction, the multi-finger action can be determined as a multi-finger swipe action.

[0255] The interaction module can also be used to control the display screen to perform a first response operation based on the first action. The first response operation can include at least one of the following response operation types: interface adjustment, target selection, playback adjustment, volume adjustment, and brightness adjustment. When the first response operation is interface adjustment, it can include switching pages, scrolling lists, moving displayed objects, adjusting the size of displayed objects, adjusting the orientation of displayed objects, or adjusting split-screen areas. When the first response operation is target selection, it can set the displayed controls, list items, icons, files, or other displayed objects corresponding to the mapped coordinates to a selected state.

[0256] The interaction module can also display a cursor on the screen based on the mapped coordinate parameters. The cursor indicates the current position of the detected object on the screen. The interaction module can update the cursor position according to the updated mapped coordinate parameters, allowing the cursor to follow the detected object on the screen. When the mapped coordinates are invalid, the detected object leaves the detection range, or the detected object is not detected temporarily, the interaction module can hide the cursor or keep the cursor at the position corresponding to the last valid mapped coordinates.

[0257] The projection module, position determination module, coordinate determination module, and interaction module described above can be implemented using virtual modules, such as software functional units or virtual devices. Alternatively, they can also be implemented using physical devices. For example, if the projection interaction device is implemented using chips, chip circuits, or hardware logic, the projection module, position determination module, coordinate determination module, and interaction module can be functional modules within integrated circuits, logic circuits, processors, or microprocessors.

[0258] The above division of units or modules is illustrative and represents only one logical functional division. In actual implementation, there may be other division methods. For example, the position determination module and the coordinate determination module can be integrated into a position processing module; the coordinate determination module and the interaction module can be integrated into an interaction processing module; the projection module can be integrated into a projection device, vehicle system, cockpit controller, or display controller.

[0259] This application also provides a projection interaction system. The projection interaction system may include a projection device, a projection medium, a position detection module, and the aforementioned projection interaction device. The projection device projects a display image onto the projection medium; the position detection module detects objects within the detection range; and the projection interaction device determines the position information and mapped coordinate parameters of the detected object based on the detection data output by the position detection module, and determines the interaction operation based on the mapped coordinate parameters.

[0260] In one implementation, the projection interaction system can be installed in a vehicle. The projection medium can be an in-vehicle projection screen, which can be installed behind the first row of seats, on the vehicle roof, on the seat backs, on the interior partition structure, or in other locations suitable for passenger viewing. The projection device can be installed on the vehicle roof, above the seats, on interior trim, on the dashboard, near the seat backs, or in other suitable projection locations.

[0261] In another possible implementation, please refer to Figure 12 , Figure 12 This is a schematic diagram of another possible structure for the projection interaction device. The projection interaction device 1200 can exemplarily be a chip or a chip system, used to implement the functions of the projection interaction device or its modules described in the foregoing embodiments. Optionally, the chip system can be composed of chips or may include chips and other discrete components.

[0262] like Figure 12As shown, the device may include at least one processor 1210, which is coupled to a memory. Optionally, the memory may be located within the projection interaction device 1200, integrated with the processor, or located outside the projection interaction device 1200. For example, the projection interaction device 1200 may also include at least one memory 1220. The at least one memory 1220 stores the necessary computer programs or instructions and / or data for implementing any of the above embodiments; the at least one processor 1210 can execute the computer programs or instructions and / or data stored in the at least one memory 1220 to complete the projection interaction method in any of the above embodiments.

[0263] Optionally, the projection interaction device 1200 also includes a communication interface 1230, through which the projection interaction device 1200 can interact with other devices. For example, the communication interface 1230 can be a transceiver, circuit, bus, module, pin, or other type of communication interface. When the projection interaction device 1200 is a chip-type device or circuit, the communication interface 1230 can also be an input / output circuit, capable of inputting, receiving, outputting, or sending information. The processor 1210 is an integrated processor, microprocessor, integrated circuit, or logic circuit, and the processor 1210 can determine the output information based on the input information.

[0264] The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 1210 can operate collaboratively with the memory 1220 and the communication interface 1230. This embodiment does not limit the connection medium between the processor 1210, the memory 1220, and the communication interface 1230.

[0265] Optionally, see Figure 12 The processor 1210, memory 1220, and communication interface 1230 are interconnected via a bus. This bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0266] In the embodiments of this application, the processor 1210 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0267] In this embodiment, the memory 1220 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). The memory 1220 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory 1220 in this embodiment can also be a circuit or any other device capable of implementing a storage function for storing program instructions and / or data.

[0268] Based on the above, this application also provides a projection interaction system, which includes a projection device, a projection medium, a position detection module, and the aforementioned projection interaction device. The projection device is used to project a display image onto the projection medium, the position detection module is used to detect a detection object located within a detection range in front of the projection medium, and the projection interaction device is used to determine mapping coordinate parameters based on the position information of the detection object, and determine the interaction operation based on the mapping coordinate parameters.

[0269] Optionally, the projection device can be connected to a vehicle infotainment system, cockpit controller, entertainment host, rear entertainment controller, display controller, mobile terminal, or other display source. The displayed screen can be a video playback interface, entertainment application interface, vehicle control interface, seat control interface, air conditioning control interface, navigation interface, or communication interface.

[0270] In a projection interaction system, the projection interaction device can perform position detection and interaction operation recognition during normal vehicle operation, as well as after the projection medium is unfolded. Position detection or response to interaction operations can be stopped when the projection medium is retracted. When the vehicle is in a designated driving state, some interaction operations can be restricted based on the vehicle's status.

[0271] Based on the foregoing, this application also provides a vehicle that includes the aforementioned projection interaction system. Exemplary examples include sedans, trucks, buses, construction vehicles, trams, gasoline-powered vehicles, new energy vehicles, or other vehicles with projection display capabilities. This application does not specifically limit the type of vehicle.

[0272] In vehicles, the vehicle projection interaction system can be used for rear-seat entertainment interaction, cockpit control interaction, video playback control, navigation display control, or other projection display interactions. The vehicle can control the display screen to update based on interactive operations, and can also control in-vehicle functions to perform corresponding operations based on interactive operations.

[0273] Based on the above, this application also provides a computer-readable storage medium storing instructions that, when executed, cause the methods provided in any of the above-described method embodiments to be implemented. The computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0274] Based on the above, this application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method provided in any of the above method embodiments.

[0275] Based on the above, this application also provides a chip, which includes at least one processing unit and an interface circuit. The interface circuit is used to provide program instructions or data to the at least one processing unit, and the at least one processing unit is used to execute the program instructions to implement the method provided in any of the above method embodiments.

[0276] Unless otherwise specified or there is a logical conflict, the terminology and / or descriptions of different implementations are consistent and can be referenced by each other. Technical features in different implementations can be combined to form new implementations based on their inherent logical relationships.

[0277] In this document, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B means: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The various numbers are for descriptive convenience only and are not used to limit the scope of protection. The sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0278] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0279] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, systems, and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0280] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0281] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A projection interaction method, characterized in that, The method includes: Control the projection device to project the display image onto the projection medium; The location detection module detects objects within the detection range and determines the location information of the detected objects. The mapping coordinate parameters are determined based on the location information. The mapping coordinate parameters include the mapping coordinates of the detected object in the display screen and the motion state information of the detected object. The motion state information of the detected object is used to represent the motion state of the detected object. The interactive operation is determined based on the mapped coordinate parameters, and the display screen is controlled to respond to the interactive operation.

2. The method according to claim 1, characterized in that, Determining the location information of the detected object specifically includes: Determine the distance between the object being detected and the position detection module; The location information of the detected object is determined based on the distance between the detected object and the location detection module.

3. The method according to claim 2, characterized in that, The position detection module further includes a driving module, which is used to control the position detection module to scan within the detection range; The location information of the detected object is determined based on the distance between the detected object and the location detection module, specifically including: The position information of the detected object is determined based on the distance and the scanning angle of the position detection module.

4. The method according to any one of claims 1-3, characterized in that, Determining the location information of the detected object specifically includes: Generate a coordinate point cloud based on the detection points corresponding to the detection object; The coordinate point cloud is filtered to determine the location information of the detected object.

5. The method according to claim 4, characterized in that, The filtering process for the coordinate point cloud includes: The coordinate point cloud is processed to filter out at least one of the background points, invalid detection points, and noise points of the projection medium.

6. The method according to any one of claims 1-5, characterized in that, Determining the mapping coordinate parameters based on the location information includes: Based on the location information of the detected object and a preset coordinate mapping relationship, the location information is converted into the mapped coordinate parameters; wherein, the preset coordinate mapping relationship is used to characterize the correspondence between the location information of the detected object and the mapped coordinates in the display screen.

7. The method according to any one of claims 1-6, characterized in that, The interactive operation is determined based on the mapped coordinate parameters, including: The first action is determined based on the mapped coordinate parameters. The first action includes at least one of the following actions: click action, double click action, long press action, swipe action, drag action, and multi-finger action. The interaction operation is determined based on the first action.

8. The method according to claim 7, characterized in that, Controlling the display screen to respond to the interactive operation specifically includes: The first action controls the display screen to perform a first response operation, which includes at least one of the following: interface adjustment, target selection, playback adjustment, volume adjustment, and brightness adjustment.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: A cursor is displayed on the display screen based on the mapped coordinate parameters, wherein the cursor is used to indicate the current position of the detected object on the display screen.

10. The method according to any one of claims 1-9, characterized in that, The detection object includes the user's finger or a tool used to point at the projection medium.

11. The method according to any one of claims 1-10, characterized in that, The detection of objects within the detection range via the position detection module specifically includes: Determine the distance between the detection object and the projection medium, and determine whether the distance between the detection object and the projection medium is within a set distance range. If it is within the set distance range, determine the position information of the detection object.

12. The method according to any one of claims 1-11, characterized in that, Determining the mapping coordinate parameters based on the location information includes: When there are multiple detection objects, the mapping coordinate parameters corresponding to each of the multiple detection objects are determined based on the location information corresponding to each of the multiple detection objects.

13. The method according to any one of claims 1-12, characterized in that, The position detection module is mounted on the transverse support beam used for winding up the projection medium.

14. A projection interaction device, characterized in that, The device includes: The projection module is used to control the projection device to project the display image onto the projection medium; The location determination module is used to detect objects within the detection range through the location detection module and determine the location information of the detected objects; The coordinate determination module is used to determine the mapping coordinate parameters based on the position information. The mapping coordinate parameters include the mapping coordinates of the detected object in the display screen and the motion state information of the detected object. The motion state information of the detected object is used to represent the motion state of the detected object. An interaction module is used to determine the interaction operation based on the mapped coordinate parameters and control the display screen to respond to the interaction operation.

15. A projection interaction system, characterized in that, It includes a projection device, a projection medium, a position detection module, and a projection interaction device as described in claim 14.

16. A vehicle, characterized in that, Including the projection interaction system as described in claim 15.

17. A projection interaction device, characterized in that, Includes a processor and a memory, wherein the processor and the memory are coupled; The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions to implement the projection interaction method as described in any one of claims 1 to 13.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed, implement the projection interaction method as described in any one of claims 1 to 13.

19. A computer program product, characterized in that, The computer program product includes instructions that, when executed, implement the projection interaction method as described in any one of claims 1 to 13.

Images