Point cloud data segmentation method, device, system and non-volatile storage medium

Abstract

The application discloses a point cloud data segmentation method, device and system and a nonvolatile storage medium. The method comprises the following steps: receiving a first space coordinate, wherein the first space coordinate is a coordinate corresponding to a target space region, the target space region is a region selected from a predetermined space region photographed by a first camera, and the first space coordinate is a coordinate in a first coordinate system corresponding to the first camera; obtaining point cloud data obtained by photographing the predetermined space region by a second camera; converting the first space coordinate into a second space coordinate in a second coordinate system corresponding to the second camera based on a conversion relationship between the first coordinate system and the second coordinate system; and segmenting the point cloud data according to the second space coordinate to obtain target data corresponding to the target space region. The application solves the technical problem that the process efficiency of cutting out point cloud data of a region of interest from complete point cloud data is low.

Description

Technical Field

[0001] This invention relates to the field of data recognition, and more specifically, to a point cloud data segmentation method, apparatus, system, and non-volatile storage medium. Background Technology

[0002] In industrial production scenarios, it is often necessary to segment specific point cloud data from the total point cloud data captured by a depth camera. However, this segmentation process is difficult to implement. For example, in 3D grasping applications, when a robotic arm automatically grasps objects on an assembly line, it needs to acquire the object's point cloud data beforehand. When a 3D camera scans the scene where the object is located, it scans not only the area where the object is located but also a large amount of background data. Therefore, it is necessary to segment the point cloud data of the area where the object is located from the total point cloud data obtained by the 3D camera. Related technologies require first establishing the coordinate system of the area to be grasped where the object is located, and then manually adjusting a large number of parameters through code to cut out the region of interest (ROI) from the full scene data of the 3D camera, that is, to cut out the point cloud data of the area where the object is located. The entire operation process is quite complex, resulting in low efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a point cloud data segmentation method, apparatus, system, and non-volatile storage medium to at least solve the technical problem of low efficiency in the process of cutting out the region of interest from complete point cloud data.

[0005] According to one aspect of the present invention, a point cloud data segmentation method is provided, comprising: receiving first spatial coordinates, wherein the first spatial coordinates are coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by a first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; acquiring point cloud data obtained by a second camera capturing the predetermined spatial region; converting the first spatial coordinates into second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera; and segmenting the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0006] Optionally, receiving the first spatial coordinates includes: receiving the first spatial coordinates sent by the augmented reality device, wherein the augmented reality device includes a first camera, the augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing a predetermined spatial area, and the first spatial coordinates are coordinates generated by the augmented reality device based on a target spatial area selected in the augmented reality scene.

[0007] Optionally, the first spatial coordinates are the coordinates of a virtual object rendered by the augmented reality device in the augmented reality scene. The virtual object is a three-dimensional graphic used to select a target spatial region in the augmented reality scene. The augmented reality device controls the virtual object to select the target spatial region in the augmented reality scene based on the first interactive action received from interacting with the virtual object.

[0008] Optionally, the representation of the first spatial coordinates matches the virtual object. When the virtual object is a sphere, the first spatial coordinates include the coordinates of the sphere's center and the radius; when the virtual object is a cuboid, the first spatial coordinates include the homogeneous coordinates of the cuboid.

[0009] Optionally, before converting the first spatial coordinates to the second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the method further includes: receiving image data obtained by the first camera from shooting a predetermined spatial area, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; calibrating the first camera and the second camera according to the image data and the point cloud data to obtain the transformation relationship between the first coordinate system and the second coordinate system.

[0010] Optionally, the method further includes: generating control instructions based on target data, wherein the control instructions are used to instruct the robotic arm to grasp the object represented by the target data, the object being located in the target space region; and sending the control instructions to the robotic arm to instruct the robotic arm to grasp the object according to the control instructions.

[0011] According to another aspect of the present invention, a point cloud data segmentation method is also provided, comprising: selecting a target spatial region in a predetermined spatial region captured by a first camera; determining a first spatial coordinate of the target spatial region in a first coordinate system corresponding to the first camera; and sending the first spatial coordinate to a control processor, wherein the control processor is configured to convert the first spatial coordinate to a second spatial coordinate in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and the control processor is further configured to segment the point cloud data obtained by the second camera capturing the predetermined spatial region according to the second spatial coordinate to obtain target data corresponding to the target spatial region.

[0012] Optionally, selecting a target spatial region within a predetermined spatial region captured by the first camera includes: generating an augmented reality scene corresponding to the predetermined spatial region based on image data obtained from the image data captured by the first camera within the predetermined spatial region; and selecting the target spatial region within the augmented reality scene.

[0013] Optionally, selecting a target spatial region in the augmented reality scene includes: receiving a first interactive action to interact with a virtual object in the augmented reality scene, wherein the virtual object is a three-dimensional graphic rendered in the augmented reality scene for selecting the target spatial region; and controlling the virtual object to select the target spatial region in the augmented reality scene according to the first interactive action.

[0014] Optionally, according to the first interactive action, controlling the virtual object to select a target spatial region in the augmented reality scene includes: responding to the first interactive action and controlling the virtual object to perform at least one of the following actions in the augmented reality scene: movement, deformation, scaling; and determining that the target spatial region is selected by the virtual object after the virtual object matches the target spatial region.

[0015] Optionally, receiving a first interactive action to interact with a virtual object in the augmented reality scene includes: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

[0016] Optionally, in response to the first interactive action, the virtual object is controlled to perform at least one of the following actions in the augmented reality scene: movement, deformation, and scaling, including: when the center point of the virtual image coincides with that of the virtual object, controlling the virtual object to move in response to a virtual action in the augmented reality scene; when the virtual image coincides with any vertex of the virtual object, responding to a virtual action to change the position of the vertex of the virtual object that coincides with the virtual image; when the entity object includes multiple objects, and the virtual images corresponding to each of the multiple objects coincide with any vertex of the virtual object and the center point of the virtual object, controlling the virtual object to respond to a virtual action to scale, wherein the virtual action includes any one of the following: the multiple virtual images increase the distance between each other, or the multiple virtual images decrease the distance between each other.

[0017] Optionally, capturing the actions of an entity object and displaying the corresponding virtual image and virtual action in an augmented reality scene includes: taking a picture of the entity object with a first camera to obtain an image of the entity object; identifying the actions of the entity object based on the image of the entity object; and rendering the virtual image and virtual action in the augmented reality scene based on the image and actions of the entity object.

[0018] Optionally, receiving a first interactive action to interact with a virtual object in an augmented reality scene includes: acquiring voice audio, wherein the voice audio is used to indicate regional features of a target spatial region; recognizing the voice audio to obtain a first interactive action to interact with the virtual object, wherein the first interactive action is used to control the virtual object to select a target spatial region with regional features.

[0019] Optionally, before receiving a first interactive action to interact with a virtual object in the augmented reality scene, the method further includes: receiving a second interactive action; and responding to the second interactive action by rendering the virtual object in the augmented reality scene.

[0020] Optionally, in response to the second interactive action, rendering a virtual object in the augmented reality scene includes: determining the virtual object type based on the second interactive action, wherein the virtual object type is used to characterize the three-dimensional shape of the virtual object; and rendering a virtual object conforming to the virtual object type in the augmented reality scene.

[0021] Optionally, determining the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera includes: representing the augmented reality scene using the first coordinate system; and determining the first spatial coordinates of the virtual object in the first coordinate system.

[0022] Optionally, the method further includes: acquiring image data of a first spatial region captured by a first camera, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; and sending the image data to a control processor, wherein the control processor determines the transformation relationship between the first coordinate system and the second coordinate system based on the image data and the point cloud data.

[0023] According to another aspect of the present invention, a point cloud data segmentation apparatus is also provided, comprising: a receiving module for receiving first spatial coordinates, wherein the first spatial coordinates are coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by a first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; an acquiring module for acquiring point cloud data obtained by a second camera capturing the predetermined spatial region; a conversion module for converting the first spatial coordinates into second spatial coordinates in the second coordinate system based on the conversion relationship between the first coordinate system and the second coordinate system corresponding to the second camera; and a segmentation module for segmenting the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0024] According to another aspect of the present invention, a point cloud data segmentation apparatus is also provided, comprising: a selection module, configured to select a target spatial region within a predetermined spatial region captured by a first camera; a determination module, configured to determine first spatial coordinates of the target spatial region in a first coordinate system corresponding to the first camera; and a transmission module, configured to transmit the first spatial coordinates to a control processor, wherein the control processor is configured to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to a transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and the control processor is further configured to segment the point cloud data obtained by the second camera capturing the predetermined spatial region according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0025] According to another aspect of the present invention, a point cloud data segmentation system is also provided, including an augmented reality (AR) device with a first camera, a control processor, and a second camera; wherein, the AR device is used to generate an augmented reality scene based on image data obtained by the first camera from a predetermined spatial region, receive a target spatial region selected in the augmented reality scene, and determine the first spatial coordinates of the target spatial region, wherein the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; the AR device is also used to send the first spatial coordinates to the control processor; the second camera is used to capture the predetermined spatial region to obtain point cloud data, and is also used to send the point cloud data to the control processor; the control processor is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and to segment target data corresponding to the target spatial region from the point cloud data according to the second spatial coordinates.

[0026] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute any of the above-described point cloud data segmentation methods.

[0027] According to another aspect of the present invention, a computer device is also provided, the computer device including a memory and a processor, the memory being used to store a program, and the processor being used to run the program stored in the memory, wherein the program executes any of the point cloud data segmentation methods described above when it runs.

[0028] In this embodiment of the invention, by receiving first spatial coordinates, wherein the first spatial coordinates are the coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by a first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; acquiring point cloud data obtained by a second camera capturing the predetermined spatial region; converting the first spatial coordinates into second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera; and segmenting the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region, the purpose of quickly cutting out point cloud data of the region of interest from complete point cloud data is achieved, thereby improving the technical effect of improving the efficiency of cutting point cloud data of the region of interest, and thus solving the technical problem of low efficiency in the process of cutting out point cloud data of the region of interest from complete point cloud data. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0030] Figure 1 A hardware structure block diagram of a computer terminal for implementing a point cloud data segmentation method is shown.

[0031] Figure 2 This is a flowchart illustrating a point cloud data segmentation method according to an embodiment of the present invention;

[0032] Figure 3 This is a flowchart illustrating a second point cloud data segmentation method according to an embodiment of the present invention;

[0033] Figure 4 This is a structural block diagram of a point cloud data segmentation device according to an embodiment of the present invention;

[0034] Figure 5 This is a structural block diagram of a point cloud data segmentation device II provided according to an embodiment of the present invention;

[0035] Figure 6 This is a structural block diagram of a point cloud data segmentation system provided according to an embodiment of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0039] A depth camera is a camera that can capture depth information of an image, such as a 3D camera. A depth camera can detect the depth distance of the shooting space, that is, it can detect the distance from each point in the image to the depth camera's camera lens.

[0040] Point cloud data refers to a set of vectors in a three-dimensional coordinate system. Scanned data is recorded in the form of points, each of which includes three-dimensional coordinates and may also include color information and reflectance information.

[0041] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. It simulates and applies computer-generated text, images, 3D models, and other virtual information to the real world, with the two types of information complementing each other to achieve "enhancement" of the real world.

[0042] According to an embodiment of the present invention, a method embodiment for point cloud data segmentation is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a point cloud data segmentation method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0044] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be implemented wholly or partially as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element in the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0045] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the point cloud data segmentation method in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the point cloud data segmentation method of the aforementioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0047] In some applications of depth cameras, it is often necessary to segment specific regions of point cloud data from the entire point cloud data captured by the depth camera. These specific regions are called Regions of Interest (ROIs). Based on the segmented ROIs, further analysis of the shape, size, and material of objects within the region can be performed. Therefore, fast and accurate point cloud segmentation is essential. To improve the efficiency of point cloud segmentation, Figure 2 The diagram illustrates a flowchart of a point cloud data segmentation method according to an embodiment of the present invention. This point cloud data segmentation method can be applied to a point cloud data processor for segmenting point cloud data. Figure 2 As shown, the method includes the following steps:

[0048] Step S202: Receive first spatial coordinates, wherein the first spatial coordinates are the coordinates corresponding to the target spatial region, the target spatial region is the region selected from the predetermined spatial region captured by the first camera, and the first spatial coordinates are the coordinates in the first coordinate system corresponding to the first camera.

[0049] In this step, the target spatial region can be one of the spatial regions captured by the depth camera. The target spatial region can be the region obtained by the user after selecting from the predetermined spatial regions, and therefore can also be called the region of interest.

[0050] Typically, users select a target spatial region from a predetermined spatial area to segment the point cloud data within that target region from the total point cloud data captured by a depth camera, thus supporting further analysis of the target spatial region. However, directly segmenting the point cloud data of the target spatial region from the point cloud data captured by a depth camera is quite difficult. Therefore, this application proposes an auxiliary method based on capturing images of the target spatial region using a first camera, which helps to quickly locate the area of ​​the target spatial region and improve the efficiency of point cloud data segmentation. Optionally, the first camera can be a 2D imaging camera that captures images of the target spatial region from an angle other than the depth camera, and the first camera may not have depth sensing capabilities.

[0051] When the first camera captures a target spatial region, the spatial coordinates of the target spatial region in a first coordinate system established with the first camera as the reference can be recorded. This first spatial coordinate is then transmitted by the first camera or a data processing device connected to the first camera to a point cloud data processor. The processor then determines the relative position of the target spatial region and the first camera based on these first spatial coordinates. Optionally, the first spatial coordinate system can be a coordinate system established with the first camera itself as the origin and the positive direction of the camera's field of view as the positive direction of the coordinate axes; alternatively, the first spatial coordinate system can be a coordinate system not with the first camera as the origin, in which the first camera is a trivial point.

[0052] As an optional embodiment, the first spatial coordinates can be received in the following manner: receiving the first spatial coordinates sent by the augmented reality device, wherein the augmented reality device includes a first camera, the augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing a predetermined spatial area, and the first spatial coordinates are coordinates generated by the augmented reality device based on the target spatial area selected in the augmented reality scene.

[0053] Optionally, the augmented reality device (hereinafter referred to as AR device) can perform the following functions: capture a predetermined spatial area, generate an augmented reality scene (hereinafter referred to as AR scene) on the display screen included in the AR device based on the image captured by the first camera, and support the user of the AR device to select a target spatial area in the AR scene. The AR device can generate a first spatial coordinate in a first spatial coordinate system according to the target spatial area selected by the user in the AR scene, and then send the first spatial coordinate to a device for processing point cloud data, such as a point cloud data processor. The AR device can include various device types, such as AR glasses or AR helmets worn by the user, or mobile terminals that are easy for the user to carry, such as smartphones or smart tablets with cameras and displays.

[0054] As an optional embodiment, the first spatial coordinates can be the coordinates of a virtual object rendered by the AR device in the AR scene. The virtual object is a three-dimensional graphic used to select a target spatial region in the AR scene. The AR device controls the virtual object to select the target spatial region in the AR scene based on the first interactive action received from interacting with the virtual object.

[0055] This optional embodiment provides an implementation method for selecting a target spatial region in an AR scene. By generating a virtual object in the AR scene as an auxiliary tool, the user can easily select the target spatial region from the AR scene by manipulating the virtual object, significantly improving the region selection efficiency. This allows the AR device to more quickly send the coordinates of the target spatial region to the point cloud data processing device to begin the point cloud data segmentation work, thus improving the efficiency of the point cloud data segmentation work. Optionally, the user can control the virtual object to cover or surround the target spatial region based on a first interactive action to select the target spatial region. The coordinates of the virtual object itself can then be used as the first spatial coordinates of the selected target spatial region. Optionally, the virtual object rendered in the AR scene can be a sphere, ellipsoid, cube, cuboid, or other types of regular or irregular geometric shapes.

[0056] As an optional embodiment, the representation of the first spatial coordinates can be matched with the virtual object. For example, if the virtual object is a sphere, the first spatial coordinates include the coordinates of the sphere's center and the radius; if the virtual object is a cuboid, the first spatial coordinates include the homogeneous coordinates of the cuboid. Using the above representation of the first spatial coordinates can reduce the amount of data for the first spatial coordinates, allowing them to be transmitted more quickly to the device used to process point cloud data.

[0057] Step S204: Obtain point cloud data obtained by the second camera capturing images of the predetermined spatial area.

[0058] The second camera can be a depth camera capable of acquiring point cloud data, such as a 3D camera. The point cloud data obtained by the second camera when shooting a predetermined spatial area can be called complete point cloud data. Since complete point cloud data includes the point cloud data of the target spatial area and the data of the background area surrounding the target spatial area, the point cloud data of the target spatial area can be segmented from the complete point cloud data of the second camera.

[0059] Step S206: Based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the first spatial coordinates are converted into second spatial coordinates in the second coordinate system.

[0060] Optionally, when the first coordinate system is established with the first camera as the center and the second coordinate system is established with the second camera as the center, the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera can characterize the relative positional relationship and lens angle relationship between the first and second cameras. This transformation relationship can be predetermined and stored in the local memory of the point cloud data processor. The relative positional relationship can represent the distance and azimuth angle between the first and second cameras, and the lens angle relationship can represent the angular relationship between the optical axis directions of the first and second cameras, i.e., the relationship between the lens directions of the first and second cameras. Based on this transformation relationship, the first spatial coordinates in the first spatial coordinate system can be converted to the second spatial coordinates in the second coordinate system.

[0061] It should be noted that the second spatial coordinates can represent the target spatial region in the second coordinate system. Therefore, the essence of the above transformation is to change the spatial position of the target spatial region from being expressed in the first coordinate system to being expressed in the second coordinate system. The second coordinate system is the coordinate system corresponding to the second camera, and therefore can be used to assist in segmenting the point cloud data of the target spatial region from the point cloud data captured by the second camera, i.e., the target data.

[0062] As an optional embodiment, before converting the first spatial coordinates to the second spatial coordinates based on the transformation relationship, the above transformation relationship can be obtained in the following way: receiving image data obtained by the first camera capturing a predetermined spatial area, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; calibrating the first camera and the second camera according to the image data and the point cloud data to obtain the transformation relationship between the first coordinate system and the second coordinate system.

[0063] The calibration between the first and second cameras can be used to determine the correspondence between their respective imaging spaces. It is understood that since the first and second cameras simultaneously capture images of a predetermined spatial area, the coordinate representations of any position within that area in the first and second spatial coordinate systems are usually different. Therefore, it is necessary to determine the relationship between the imaging spaces of the first and second cameras beforehand through camera calibration, and thus determine the transformation relationship between the first and second coordinate systems.

[0064] Step S208: Segment the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region. In this step, since the second spatial coordinates are the coordinates that represent the target spatial region in the second coordinate system, and the point cloud data is the complete point cloud data obtained by the second camera capturing the predetermined spatial region, the point cloud data within the area corresponding to the second spatial coordinates can be segmented from the complete point cloud data to obtain the target data corresponding to the target spatial region, thus achieving the technical objective of point cloud data segmentation.

[0065] Through the above steps, the goal of quickly cutting out the region of interest from the complete point cloud data is achieved, thereby improving the technical efficiency of cutting out the region of interest and solving the technical problem of low efficiency in the process of cutting out the region of interest from the complete point cloud data.

[0066] As an optional embodiment, the above method may further include the following steps: generating control instructions based on target data, wherein the control instructions are used to instruct the robotic arm to grasp the object represented by the target data, the object being located in the target space region; and sending control instructions to the robotic arm to instruct the robotic arm to grasp the object according to the control instructions.

[0067] The technical solution provided in this optional embodiment can be applied to robotic arm manipulation scenarios, such as manipulating a robotic arm to grasp an object in a target spatial region. In some application scenarios, the object to be grasped may be placed on an assembly line or near the robotic arm. When manipulating the robotic arm to grasp the object, in order to determine the appropriate hand shape, force, and movement direction for the robotic arm to grasp the object, the three-dimensional point cloud data of the object to be grasped can be obtained first. Then, the three-dimensional point cloud data of the object to be grasped can be input into the robotic arm control program to generate control instructions for the robotic arm to grasp the object. Therefore, based on the above embodiment or optional embodiment, the point cloud data of the target spatial region can be segmented from the predetermined spatial region to obtain target data corresponding to the target spatial region. Then, based on the target data, one or more of the object's shape, contour, size, and material can be determined, thereby generating control instructions for grasping the object. This allows the robotic arm to use a more accurate and effective hand shape and force when grasping the object according to the control instructions, improving the reliability of the robotic arm when performing the grasping action.

[0068] Figure 3This is a flowchart illustrating a second point cloud data segmentation method according to an embodiment of the present invention. Optionally, the second point cloud data segmentation method can be applied to an AR device with augmented reality functionality. The AR device can be connected to a first camera, or the first camera can be pre-installed in the AR device. The AR device can acquire images captured by the first camera and perform subsequent data processing based on these images. Further, the AR device can send the first spatial coordinates to a point cloud data processor for segmenting point cloud data, such as the control processor in step S306 below. Therefore, the AR device can have wired or wireless communication capabilities to facilitate data transmission to the control processor. Figure 3 As shown, the above-mentioned point cloud data segmentation method two may include the following steps:

[0069] Step S302: Select the target spatial region from the predetermined spatial region captured by the first camera.

[0070] In this step, the first camera can be a camera pre-installed on the AR device or a camera pre-installed on a device with AR functionality. The AR device can include AR helmets or AR glasses, and the device with AR functionality can be a portable mobile terminal, such as a smartphone or smart tablet with a camera and display (hereinafter collectively referred to as an AR device). Compared to the second camera, the AR device is more flexible and portable, thus allowing users to carry it with them. Users can use the first camera in the AR device to take pictures around the target spatial area to obtain an image of the predetermined spatial area. The predetermined spatial area includes the target spatial area and the background area simultaneously captured by the first camera when the target spatial area is photographed. Furthermore, users can select the target spatial area from the predetermined spatial area through interaction with the AR device. The entire process is simple to operate, responsive, and easy for users to master.

[0071] To further improve the intelligence of users selecting target areas from predetermined spatial regions, one or more optional embodiments can be adopted to improve the user's operation process for selecting target spatial regions. As an optional embodiment, the user can be supported in selecting a target spatial region from a predetermined spatial region captured by a first camera in the following way: generating an augmented reality scene corresponding to the predetermined spatial region based on the image data obtained by the first camera capturing the predetermined spatial region; and selecting the target spatial region in the augmented reality scene.

[0072] In this optional embodiment, the AR device can generate an AR scene based on image data captured by a first camera. Users can visually view a predetermined spatial area within the AR scene and select a target spatial area from within it. For example, the area where an object is located can be identified as the target spatial area and selected. Optionally, the AR device can include a display screen to display the AR scene. The AR device can also respond to a user's selection command by selecting a target spatial area within the AR scene on the display screen. For example, selecting an object within the AR scene and identifying the area where the object is located as the target spatial area.

[0073] Optionally, this application may also provide an embodiment for selecting a target spatial region in an augmented reality scene as follows: receiving a first interaction action to interact with a virtual object in the augmented reality scene, wherein the virtual object is a three-dimensional graphic rendered in the augmented reality scene for selecting a target spatial region; and controlling the virtual object to select the target spatial region in the augmented reality scene according to the first interaction action.

[0074] In this optional embodiment, the virtual object can be a 3D graphic rendered in the AR scene, and the first interactive action can be an action performed by the user interacting with the AR device. As an optional embodiment, controlling the virtual object based on the first interactive action may include the following steps: responding to the first interactive action, controlling the virtual object to perform at least one of the following actions in the augmented reality scene: movement, deformation, or scaling; after the virtual object matches a target spatial region, determining that the target spatial region has been selected through the virtual object. The first interactive action can be used to control the virtual object in the AR scene to move, enlarge, shrink, or deform, thereby enabling the changed virtual object to select a target spatial region in the AR scene. Optionally, the selection of the target spatial region by the virtual object can be defined as the spatial range where the virtual object is located in the AR scene.

[0075] As a specific embodiment, in an application scenario where a user controls a robotic arm to grasp an object using an AR device, a virtual object with the same or similar shape to the object to be grasped can be generated in the AR scene. By responding to the user's first interactive action, the virtual object can be controlled to overlap with the object to be grasped, or the virtual object can be controlled to occupy a slightly larger space in the AR scene than the object to be grasped, achieving the effect of enveloping the object. The space occupied by the virtual object can then be defined as the target spatial region. The coordinates of this target spatial region are sent to the control processor for segmenting point cloud data. This allows the control processor to segment the point cloud data corresponding to the target spatial region from the point cloud data acquired by the second camera, obtaining target data. Therefore, the target data can be used to characterize the shape, size, material, or a combination of these characteristics of the object to be grasped within the target spatial region.

[0076] As an optional embodiment, before receiving the first interactive action to interact with the virtual object in the augmented reality scene, a second interactive action can be received first, and then the virtual object can be rendered in the augmented reality scene in response to the second interactive action.

[0077] As an optional embodiment, rendering a virtual object in an augmented reality scene in response to a second interactive action includes: determining a virtual object type based on the second interactive action, wherein the virtual object type is used to characterize the three-dimensional shape of the virtual object; and rendering a virtual object conforming to the virtual object type in the augmented reality scene.

[0078] Optionally, the type of virtual object can be selected by the user based on the characteristics of the object to be grasped. The user can select the type of virtual object through a second interactive action and create the virtual object in the AR scene. The AR scene can provide users with various types of virtual objects such as spheres, cuboids, cubes, or cylinders. Users can generate appropriate virtual objects based on the shape characteristics of the object to be grasped. For example, if the object to be grasped is a basketball, using a spherical area in the target space can cover the object with minimal redundancy, so a spherical virtual object can be generated in the AR scene.

[0079] As an optional embodiment, when receiving the first interactive action to interact with a virtual object in an augmented reality scene, it can be implemented in the following optional way: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

[0080] It should be noted that the physical object can be a part of the user's body operating the AR device, or an external control aid connected to the AR device. The characteristic of a physical object is that it exists in the real world, and actions based on the physical object can represent the user's intention to manipulate virtual objects in the AR scene. Optionally, the physical object can be a user's hand, foot, forearm, etc., or it can be a control handle or control bracelet of the AR device. When the physical object is the user's hand, the AR device can sense and recognize the user's hand movements, and create, match, and manipulate virtual objects in the AR scene based on specific hand movements. For example, when the AR device recognizes that the user's fist clenching time exceeds a predetermined threshold, it can recognize this action as a virtual object creation action, and then create a virtual object in the AR scene.

[0081] For example, a virtual image corresponding to a physical object can be displayed in an AR scene, such as a virtual hand image. This virtual hand image can respond to the user's hand movements and perform the same actions as the user's hand in the AR scene. In this way, users can change the actions of the physical object and thus change the actions of the corresponding virtual image in the AR scene, thereby achieving more flexible manipulation of virtual objects in the AR scene.

[0082] As an optional embodiment, when capturing the actions of an entity object and displaying the corresponding virtual image and virtual action in an augmented reality scene, the following steps can be taken: using a first camera to photograph the entity object to obtain an image of the entity object; identifying the actions of the entity object based on the image of the entity object; and rendering virtual images and virtual actions in the augmented reality scene based on the image and actions of the entity object.

[0083] As an optional implementation, the matching rules between virtual images and virtual objects can be used to determine which virtual object in the AR scene the user wishes to manipulate. For example, if the virtual object is a virtual sphere in the AR scene, the user can control the virtual image corresponding to the physical object to move to the center of the virtual sphere through the actions of the physical object. If the user further controls the physical object to perform a selection action, the virtual image can simultaneously perform a virtual selection action. Combining the fact that the virtual image is at the center of the virtual sphere and that the virtual image has performed a virtual selection action, it can be determined that the virtual image and the virtual object (virtual sphere) satisfy the matching rules, thus confirming that the physical object and the virtual object are successfully matched. Subsequent actions of the physical object (including the virtual actions corresponding to the physical object) can be identified as the first interactive action to manipulate the virtual object, and the AR device can respond to the first interactive action to manipulate the virtual object in the AR scene.

[0084] As an optional embodiment, the AR device can respond to a first interactive action to control a virtual object in an augmented reality scene to perform at least one of the following actions: movement, deformation, scaling. The process of manipulating the virtual object includes: when the center point of the virtual image coincides with that of the virtual object, controlling the virtual object to move in response to a virtual action in the augmented reality scene; when the virtual image coincides with any vertex of the virtual object, responding to a virtual action to change the position of the vertex of the virtual object that coincides with the virtual image; when the entity object includes multiple objects, and the virtual images corresponding to each of the multiple objects coincide with any vertex of the virtual object and the center point of the virtual object, controlling the virtual object to respond to a virtual action to scale, wherein the virtual action includes any one of the following: the multiple virtual images increase the distance between each other, or the multiple virtual images decrease the distance between each other.

[0085] It should be noted that scenarios where the entity object is multiple objects can include: the entity object being the user's hand, and the user is detected to be operating the virtual object with both hands simultaneously; or, the entity object being an AR controller, and the user is detected to be operating the virtual object with at least two AR controllers simultaneously.

[0086] Optionally, users can select objects by clenching their fists or by pressing specific buttons on the AR controller. When a user moves multiple virtual images corresponding to multiple physical objects to any vertex and center point of a virtual object while simultaneously making a selection gesture, the virtual object's scaling function can be activated. Reducing the distance between the user's hands indicates shrinking the virtual object's size, while increasing the distance indicates enlarging it. The principle of controlling the AR controller's scaling is the same as controlling the hand gestures. Through these interactive methods, users can conveniently manipulate virtual objects in the AR scene, enabling the selection of target spatial regions. For example, selecting the spatial range of an object to be grasped involves placing a virtual object over the object in the AR scene, defining the corresponding spatial range as the target spatial region, and sending the coordinates of the target spatial region to the point cloud segmentation processor. The processor then segments the point cloud data within the range corresponding to the target spatial region's coordinates to obtain the point cloud data of the object to be grasped. This operation process is simple and smooth, significantly accelerating the efficiency of point cloud data segmentation.

[0087] As an optional embodiment, the first interactive action received when interacting with a virtual object in the augmented reality scene can also be performed via voice. When interacting via voice, audio can be first acquired, wherein the audio is used to indicate the regional features of the target spatial region; then the audio is recognized to obtain the first interactive action for interacting with the virtual object, wherein the first interactive action is used to control the virtual object to select the target spatial region with regional features.

[0088] In this optional embodiment, the AR device may include a sound acquisition unit for acquiring the user's voice. The AR device's processor may also perform natural language processing to identify the user's intent from the voice. For example, if the user voices "Create a virtual object," the AR device may respond by creating a new virtual object in the AR scene. Alternatively, the user may voice "Select a sphere in the AR scene." The AR device may analyze the user's audio and determine that the target spatial region the user wishes to select is the spatial region occupied by a spherical object in the AR scene. This generates a first interactive action, which controls the virtual object to transform into a virtual sphere and move to the spatial region occupied by the spherical object in the AR scene, thus selecting the target spatial region. This optional embodiment provides a novel region selection interaction method, which can further accelerate the rate at which the user selects the region to be segmented in point cloud segmentation, improving the intelligence level of point cloud segmentation.

[0089] Step S304: Determine the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera.

[0090] As an optional embodiment, when determining the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera, the first coordinate system can be used to represent the augmented reality scene; the first spatial coordinates of the virtual object in the first coordinate system can also be determined. The first coordinate system can be a Cartesian coordinate system or a polar coordinate system with the location of the first camera as the origin.

[0091] Step S306: Send the first spatial coordinates to the control processor. The control processor is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera. The control processor is also used to segment the point cloud data obtained by the second camera from the predetermined spatial area according to the second spatial coordinates to obtain target data corresponding to the target spatial area.

[0092] Through the above steps, the goal of helping users quickly extract the region of interest from complete point cloud data is achieved, thereby improving the technical efficiency of extracting the region of interest from complete point cloud data and solving the technical problem of low efficiency in the process of extracting the region of interest from complete point cloud data.

[0093] As an optional embodiment, the above method may further include the following steps: acquiring image data of a first spatial region captured by a first camera, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; sending the image data to a control processor, wherein the control processor determines the transformation relationship between a first coordinate system and a second coordinate system based on the image data and the point cloud data. This optional embodiment can reduce the processing power of the AR device's processor. The AR device does not need to complete the coordinate system establishment and target spatial region coordinate generation locally, but can directly send the image data to the background control processor, avoiding excessive computational load on the AR device that would slow down the overall segmentation of the point cloud data.

[0094] The following describes a specific optional embodiment based on the above method. In this optional embodiment, the entire point cloud data segmentation system includes an AR glasses (equivalent to an AR device including a first camera), an RGBD camera (a depth camera, abbreviated as RGBD camera, equivalent to a second camera), and a main control system (equivalent to a control processor for point cloud segmentation). The main control system includes communication nodes, coordinate system management nodes, gesture detection nodes, rendering nodes, and point cloud segmentation nodes for implementing the above-described point cloud ROI segmentation method based on AR functionality. Specifically, the process of implementing point cloud segmentation may include the following steps:

[0095] Step 1: The communication nodes of the main control system establish communication with the RGBD camera and AR glasses respectively.

[0096] Step 2: Register the scene captured by the RGBD camera and AR glasses. The coordinate system management node establishes the coordinate system association between the coordinate system of the RGBD camera and the coordinate system of the AR glasses. Based on the coordinate system association, the coordinates in the RGBD camera coordinate system can be transformed to the coordinate system of the AR glasses.

[0097] Step 3: The gesture detection node detects the user's gestures in real time (gestures are equivalent to physical objects). The user creates a unit cube (equivalent to a virtual object) with a side length of 2 decimeters centered on the fist by clenching their fist and holding it still for 2 seconds. The rendering node then generates this cube in the AR glasses.

[0098] Step 4: The user edits the cube through interactive actions. For example, by aligning their finger with a vertex of the cube for 2 seconds, the user can enter edit mode and drag the vertex multiple times to edit the original cube into a Region of Interest (ROI, which is equivalent to the target space region).

[0099] Step 5: The coordinate system management node converts the coordinates of the generated spatial ROI in the AR glasses coordinate system to the coordinates in the RGBD camera coordinate system.

[0100] Step 6: The point cloud segmentation node receives the spatial coordinates of the ROI under the RGBD camera system, and segments the point cloud data of the entire scene based on the spatial coordinates collected by the RGBD camera system to extract the point cloud data of the ROI region.

[0101] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the point cloud data segmentation method according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0103] According to an embodiment of the present invention, a point cloud data segmentation apparatus for implementing the above-described point cloud data segmentation method is also provided. Figure 4 This is a structural block diagram of a point cloud data segmentation device according to an embodiment of the present invention, such as... Figure 4 As shown, the point cloud data segmentation device includes: a receiving module 42, an acquisition module 44, a conversion module 46, and a segmentation module 48. The point cloud data segmentation device will be described below.

[0104] The receiving module 42 is used to receive first spatial coordinates, wherein the first spatial coordinates are the coordinates corresponding to the target spatial region, the target spatial region is the region selected from the predetermined spatial region captured by the first camera, and the first spatial coordinates are the coordinates in the first coordinate system corresponding to the first camera.

[0105] The acquisition module 44 is connected to the receiving module 42 and is used to acquire point cloud data obtained by the second camera from shooting a predetermined spatial area.

[0106] The conversion module 46, connected to the acquisition module 44, is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system based on the conversion relationship between the first coordinate system and the second coordinate system corresponding to the second camera.

[0107] The segmentation module 48, connected to the conversion module 46, is used to segment the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0108] It should be noted that the receiving module 42, acquiring module 44, conversion module 46, and segmentation module 48 mentioned above correspond to steps S202 to S208 in the embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in the embodiments.

[0109] According to an embodiment of the present invention, a second point cloud data segmentation apparatus for implementing the second point cloud data segmentation method described above is also provided. Figure 5 This is a structural block diagram of a point cloud data segmentation device two provided according to an embodiment of the present invention, as shown below. Figure 5 As shown, the point cloud data segmentation device 2 includes: a selection module 52, a determination module 54, and a transmission module 56. The device will be described below.

[0110] Selection module 52 is used to select a target spatial region within a predetermined spatial region captured by the first camera.

[0111] The determination module 54, connected to the selection module 52, is used to determine the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera.

[0112] The sending module 56, connected to the determining module 54, is used to send the first spatial coordinates to the control processor. The control processor is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera. The control processor is also used to segment the point cloud data obtained by the second camera from the predetermined spatial area according to the second spatial coordinates to obtain target data corresponding to the target spatial area.

[0113] It should be noted that the selection module 52, the determination module 54, and the sending module 56 mentioned above correspond to steps S302 to S306 in the embodiments. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in the embodiments.

[0114] According to embodiments of the present invention, a point cloud data segmentation system for implementing the above-described point cloud data segmentation method is also provided. Figure 6 This is a structural block diagram of a point cloud data segmentation system provided according to an embodiment of the present invention, such as... Figure 6As shown, the point cloud data segmentation system includes: an AR device 64 including a first camera 62, a control processor 66, and a second camera 68. The system will be described below.

[0115] AR device 64 is used to generate an augmented reality scene based on image data obtained from a predetermined spatial area captured by first camera 62, receive a selected target spatial area in the augmented reality scene, and determine the first spatial coordinates of the target spatial area, wherein the first spatial coordinates are coordinates in a first coordinate system corresponding to first camera 62; AR device 64 is also used to send the first spatial coordinates to control processor 66; second camera 68 is used to capture a predetermined spatial area to obtain point cloud data, and is also used to send the point cloud data to control processor 66; control processor 66 is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to second camera 68, and to segment target data corresponding to the target spatial area from the point cloud data according to the second spatial coordinates.

[0116] It should be noted that the point cloud data segmentation system and the corresponding point cloud data segmentation method described above are implemented in the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run on the computer terminal 10 provided in the embodiments.

[0117] Embodiments of the present invention may provide a computer device. Optionally, in this embodiment, the computer device may be located in at least one of a plurality of network devices in a computer network. The computer device includes a memory and a processor.

[0118] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the point cloud data segmentation method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned point cloud data segmentation method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0119] The processor can invoke information and application programs stored in the memory through the transmission device to perform the following steps: receiving first spatial coordinates, wherein the first spatial coordinates are the coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by the first camera, and the first spatial coordinates are the coordinates in a first coordinate system corresponding to the first camera; acquiring point cloud data obtained by the second camera capturing the predetermined spatial region; converting the first spatial coordinates into second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera; and segmenting the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0120] Optionally, the processor may also execute program code for the following steps: receiving first spatial coordinates, including: receiving first spatial coordinates sent by an augmented reality device, wherein the augmented reality device includes a first camera, the augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing a predetermined spatial area, and the first spatial coordinates are coordinates generated by the augmented reality device based on a target spatial area selected in the augmented reality scene.

[0121] Optionally, the processor may also execute program code that performs the following steps: the first spatial coordinates are the coordinates of a virtual object rendered by the augmented reality device in the augmented reality scene, the virtual object is a three-dimensional graphic used to select a target spatial region in the augmented reality scene, and the augmented reality device controls the virtual object to select the target spatial region in the augmented reality scene according to the first interactive action received from interacting with the virtual object.

[0122] Optionally, the processor may also execute program code that includes the following steps: the representation of the first spatial coordinates matches the virtual object; when the virtual object is a sphere, the first spatial coordinates include the coordinates of the sphere's center and the radius; when the virtual object is a cuboid, the first spatial coordinates include the homogeneous coordinates of the cuboid.

[0123] Optionally, the processor may also execute program code for the following steps: before converting the first spatial coordinates to second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the method further includes: receiving image data obtained by the first camera from a predetermined spatial area, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; calibrating the first camera and the second camera according to the image data and the point cloud data to obtain the transformation relationship between the first coordinate system and the second coordinate system.

[0124] Optionally, the processor may also execute program code that performs the following steps: generating control instructions based on target data, wherein the control instructions are used to instruct the robotic arm to grasp the object represented by the target data, the object being located in the target space region; and sending the control instructions to the robotic arm to instruct the robotic arm to grasp the object according to the control instructions.

[0125] Optionally, the processor may also execute program code for the following steps: selecting a target spatial region within a predetermined spatial region captured by the first camera; determining the first spatial coordinates of the target spatial region in a first coordinate system corresponding to the first camera; and sending the first spatial coordinates to the control processor. The control processor is configured to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera. The control processor is also configured to segment the point cloud data obtained by the second camera from the predetermined spatial region based on the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0126] Optionally, the processor may also execute program code that performs the following steps: selecting a target spatial region within a predetermined spatial region captured by the first camera, including: generating an augmented reality scene corresponding to the predetermined spatial region based on image data obtained from the image data captured by the first camera within the predetermined spatial region; and selecting the target spatial region within the augmented reality scene.

[0127] Optionally, the processor may also execute program code that performs the following steps: selecting a target spatial region in an augmented reality scene, including: receiving a first interaction action to interact with a virtual object in the augmented reality scene, wherein the virtual object is a three-dimensional graphic rendered in the augmented reality scene for selecting the target spatial region; and controlling the virtual object to select the target spatial region in the augmented reality scene according to the first interaction action.

[0128] Optionally, the processor may also execute program code that performs the following steps: controlling a virtual object to select a target spatial region in an augmented reality scene based on a first interactive action, including: responding to the first interactive action and controlling the virtual object to perform at least one of the following actions in the augmented reality scene: movement, deformation, scaling; and determining that the target spatial region is selected by the virtual object after the virtual object matches the target spatial region.

[0129] Optionally, the processor may also execute program code that performs the following steps: receiving a first interactive action to interact with a virtual object in an augmented reality scene, including: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

[0130] Optionally, the processor may also execute program code that performs the following steps: in response to a first interactive action, controls a virtual object to perform at least one of the following actions in an augmented reality scene: movement, deformation, or scaling, including: when the center point of a virtual image coincides with that of a virtual object, controlling the virtual object to move in response to a virtual action in the augmented reality scene; when the virtual image coincides with any vertex of a virtual object, in response to a virtual action, changing the position of the vertex in the virtual object that coincides with the virtual image; when the entity object includes multiple objects, and the virtual images corresponding to each of the multiple objects coincide with any vertex of the virtual object and the center point of the virtual object, controlling the virtual object to perform scaling in response to a virtual action, wherein the virtual action includes any one of the following: the multiple virtual images increase the distance between each other, or the multiple virtual images decrease the distance between each other.

[0131] Optionally, the processor may also execute program code that performs the following steps: capturing the action of an entity object and displaying the virtual image and virtual action corresponding to the entity object in an augmented reality scene, including: taking a picture of the entity object with a first camera to obtain an image of the entity object; identifying the action of the entity object based on the image of the entity object; and rendering the virtual image and virtual action in the augmented reality scene based on the image and action of the entity object.

[0132] Optionally, the processor may also execute program code that performs the following steps: receiving a first interactive action to interact with a virtual object in an augmented reality scene, including: acquiring voice audio, wherein the voice audio is used to indicate the regional features of a target spatial region; recognizing the voice audio to obtain the first interactive action to interact with the virtual object, wherein the first interactive action is used to control the virtual object to select a target spatial region with regional features.

[0133] Optionally, the processor may also execute program code that performs the following steps: before receiving a first interactive action to interact with a virtual object in the augmented reality scene, the method further includes: receiving a second interactive action; and in response to the second interactive action, rendering the virtual object in the augmented reality scene.

[0134] Optionally, the processor may also execute program code that performs the following steps: responding to a second interactive action and rendering a virtual object in an augmented reality scene, including: determining a virtual object type based on the second interactive action, wherein the virtual object type is used to characterize the three-dimensional shape of the virtual object; and rendering a virtual object that conforms to the virtual object type in the augmented reality scene.

[0135] Optionally, the processor may also execute program code that performs the following steps: determining the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera, including: representing the augmented reality scene using the first coordinate system; and determining the first spatial coordinates of the virtual object in the first coordinate system.

[0136] Optionally, the processor may also execute program code for the following steps: acquiring image data of a first spatial region captured by a first camera, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; and sending the image data to a control processor, wherein the control processor determines the transformation relationship between the first coordinate system and the second coordinate system based on the image data and the point cloud data.

[0137] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0138] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the point cloud data segmentation method provided in the above embodiments.

[0139] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0140] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving first spatial coordinates, wherein the first spatial coordinates are coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by the first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; acquiring point cloud data obtained by the second camera capturing the predetermined spatial region; converting the first spatial coordinates into second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera; and segmenting the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0141] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving first spatial coordinates, including: receiving first spatial coordinates sent by an augmented reality device, wherein the augmented reality device includes a first camera, the augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing a predetermined spatial area, and the first spatial coordinates are coordinates generated by the augmented reality device based on a target spatial area selected in the augmented reality scene.

[0142] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the first spatial coordinates are the coordinates of a virtual object rendered by the augmented reality device in the augmented reality scene, the virtual object is a three-dimensional graphic used to select a target spatial region in the augmented reality scene, and the augmented reality device controls the virtual object to select the target spatial region in the augmented reality scene according to the first interactive action received from interacting with the virtual object.

[0143] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: the representation of the first spatial coordinates matches the virtual object; when the virtual object is a sphere, the first spatial coordinates include the coordinates of the sphere's center and the radius; when the virtual object is a cuboid, the first spatial coordinates include the homogeneous coordinates of the cuboid.

[0144] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before converting the first spatial coordinates to second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the method further includes: receiving image data obtained by the first camera capturing a predetermined spatial area, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; calibrating the first camera and the second camera according to the image data and the point cloud data to obtain the transformation relationship between the first coordinate system and the second coordinate system.

[0145] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: generating control instructions based on target data, wherein the control instructions are used to instruct the robotic arm to grasp the object represented by the target data, the object being located in the target space region; and sending control instructions to the robotic arm to instruct the robotic arm to grasp the object according to the control instructions.

[0146] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: selecting a target spatial region within a predetermined spatial region captured by the first camera; determining the first spatial coordinates of the target spatial region in a first coordinate system corresponding to the first camera; and sending the first spatial coordinates to the control processor, wherein the control processor is configured to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and the control processor is further configured to segment the point cloud data obtained by the second camera capturing the predetermined spatial region according to the second spatial coordinates to obtain target data corresponding to the target spatial region.

[0147] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: selecting a target spatial region in a predetermined spatial region captured by the first camera, including: generating an augmented reality scene corresponding to the predetermined spatial region based on image data obtained from the image data captured by the first camera in the predetermined spatial region; and selecting the target spatial region in the augmented reality scene.

[0148] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: selecting a target spatial region in an augmented reality scene, including: receiving a first interaction action to interact with a virtual object in the augmented reality scene, wherein the virtual object is a three-dimensional graphic rendered in the augmented reality scene for selecting the target spatial region; and controlling the virtual object to select the target spatial region in the augmented reality scene according to the first interaction action.

[0149] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: controlling a virtual object to select a target spatial region in an augmented reality scene according to a first interactive action, including: responding to the first interactive action, controlling the virtual object to perform at least one of the following actions in the augmented reality scene: movement, deformation, scaling; and determining that the target spatial region is selected by the virtual object after the virtual object matches the target spatial region.

[0150] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving a first interactive action to interact with a virtual object in an augmented reality scene, including: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

[0151] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: in response to a first interactive action, controlling a virtual object to perform at least one of the following actions in an augmented reality scene: movement, deformation, and scaling, including: when the center point of the virtual image coincides with that of the virtual object, controlling the virtual object to move in response to a virtual action in the augmented reality scene; when the virtual image coincides with any vertex of the virtual object, in response to a virtual action, changing the position of the vertex in the virtual object that coincides with the virtual image; when the entity object includes multiple objects, and the virtual images corresponding to each of the multiple objects coincide with any vertex of the virtual object and the center point of the virtual object, controlling the virtual object to scale in response to a virtual action, wherein the virtual action includes any one of the following: the multiple virtual images increasing the distance between each other, or the multiple virtual images decreasing the distance between each other.

[0152] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: capturing the action of an entity object and displaying the virtual image and virtual action corresponding to the entity object in an augmented reality scene, including: taking a picture of the entity object with a first camera to obtain an image of the entity object; identifying the action of the entity object based on the image of the entity object; and rendering the virtual image and virtual action in the augmented reality scene based on the image of the entity object and the action of the entity object.

[0153] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving a first interactive action to interact with a virtual object in an augmented reality scene, including: acquiring voice audio, wherein the voice audio is used to indicate regional features of a target spatial region; recognizing the voice audio to obtain the first interactive action to interact with the virtual object, wherein the first interactive action is used to control the virtual object to select a target spatial region with regional features.

[0154] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: before receiving a first interactive action to interact with a virtual object in the augmented reality scene, the method further includes: receiving a second interactive action; and in response to the second interactive action, rendering a virtual object in the augmented reality scene.

[0155] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: rendering a virtual object in an augmented reality scene in response to a second interactive action, including: determining a virtual object type based on the second interactive action, wherein the virtual object type is used to characterize the three-dimensional shape of the virtual object; and rendering a virtual object conforming to the virtual object type in the augmented reality scene.

[0156] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera, including: representing the augmented reality scene using the first coordinate system; and determining the first spatial coordinates of the virtual object in the first coordinate system.

[0157] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: acquiring image data obtained by the first camera from a first spatial region, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; sending the image data to the control processor, wherein the control processor determines the transformation relationship between the first coordinate system and the second coordinate system based on the image data and the point cloud data.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0164] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A point cloud data segmentation method, characterized in that, include: Receive first spatial coordinates, wherein the first spatial coordinates are coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by the first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera; Obtain point cloud data from the second camera capturing the predetermined spatial area; Based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the first spatial coordinates are converted into second spatial coordinates in the second coordinate system; The point cloud data is segmented according to the second spatial coordinates to obtain target data corresponding to the target spatial region; The step of receiving the first spatial coordinates includes: receiving the first spatial coordinates sent by an augmented reality device, wherein the augmented reality device includes a first camera, and the augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing the predetermined spatial area. The first spatial coordinates are coordinates generated by the augmented reality device based on the target spatial area selected in the augmented reality scene. The target spatial area is selected by receiving a first interactive action that interacts with a virtual object in the augmented reality scene. The first interactive action is obtained by: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

2. The method according to claim 1, characterized in that, The first spatial coordinates are the coordinates of a virtual object rendered by the augmented reality device in the augmented reality scene. The virtual object is a three-dimensional graphic used to select the target spatial region in the augmented reality scene. The augmented reality device controls the virtual object to select the target spatial region in the augmented reality scene according to a first interactive action received from interacting with the virtual object.

3. The method according to claim 2, characterized in that, The representation of the first spatial coordinates matches that of the virtual object. When the virtual object is a sphere, the first spatial coordinates include the coordinates of the center of the sphere and the radius; when the virtual object is a cuboid, the first spatial coordinates include the homogeneous coordinates of the cuboid.

4. The method according to claim 1, characterized in that, Before converting the first spatial coordinates to second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, the method further includes: Receive image data obtained by the first camera from the predetermined spatial area, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; The first camera and the second camera are calibrated based on the image data and the point cloud data to obtain the transformation relationship between the first coordinate system and the second coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Control instructions are generated based on the target data, wherein the control instructions are used to instruct the robotic arm to grasp the object represented by the target data, and the object is located in the target space region; The control command is sent to the robotic arm, instructing the robotic arm to grasp the object according to the control command.

6. A point cloud data segmentation method, characterized in that, include: Select the target spatial region within the predetermined spatial region captured by the first camera; Determine the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera; The first spatial coordinates are sent to the control processor, wherein the control processor is configured to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and the control processor is further configured to segment the point cloud data obtained by the second camera from the predetermined spatial area according to the second spatial coordinates to obtain target data corresponding to the target spatial area. The target space region is selected by receiving a first interactive action that interacts with a virtual object in the augmented reality scene. The first interactive action is obtained by: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

7. The method according to claim 6, characterized in that, Selecting the target spatial region from the predetermined spatial region captured by the first camera includes: Based on the image data obtained by the first camera capturing the predetermined spatial region, an augmented reality scene corresponding to the predetermined spatial region is generated; Select the target spatial region in the augmented reality scene.

8. The method according to claim 7, characterized in that, Selecting the target spatial region in the augmented reality scene includes: Receive a first interactive action to interact with a virtual object in the augmented reality scene, wherein the virtual object is a three-dimensional graphic rendered in the augmented reality scene for selecting the target spatial region; Based on the first interactive action, the virtual object is controlled to select the target space region in the augmented reality scene.

9. The method according to claim 8, characterized in that, The step of controlling the virtual object to select the target spatial region in the augmented reality scene according to the first interactive action includes: In response to the first interactive action, the virtual object in the augmented reality scene is controlled to perform at least one of the following actions: movement, deformation, and scaling. After the virtual object matches the target spatial region, it is determined that the target spatial region has been selected through the virtual object.

10. The method according to claim 6, characterized in that, In response to the first interactive action, the virtual object in the augmented reality scene is controlled to perform at least one of the following actions: movement, deformation, and scaling, including: When the center point of the virtual image coincides with that of the virtual object, the virtual object is controlled to move in the augmented reality scene in response to the virtual action; If the virtual image coincides with any vertex of the virtual object, in response to the virtual action, the position of the vertex in the virtual object that coincides with the virtual image is changed; When the entity object includes multiple objects, and the virtual image corresponding to each of the multiple objects coincides with any vertex of the virtual object and the center point of the virtual object, the virtual object is controlled to respond to the virtual action to scale. The virtual action includes any one of the following: the multiple virtual images increase the distance between each other, or the multiple virtual images decrease the distance between each other.

11. The method according to claim 6, characterized in that, The process of capturing the actions of an entity object and displaying the corresponding virtual image and virtual actions of the entity object in the augmented reality scene includes: The first camera is used to capture an image of the physical object, thereby obtaining an image of the physical object; Based on the image of the entity object, the action of the entity object is identified; Based on the image and actions of the entity object, the virtual image and virtual actions are rendered in the augmented reality scene.

12. The method according to claim 8, characterized in that, The first interactive action of receiving and interacting with virtual objects in the augmented reality scene includes: Acquire speech audio, wherein the speech audio is used to indicate the regional characteristics of the target spatial region; The voice audio is recognized to obtain a first interactive action for interacting with the virtual object, wherein the first interactive action is used to control the virtual object to select the target spatial region with the regional characteristics.

13. The method according to claim 8, characterized in that, Prior to receiving the first interactive action of interacting with virtual objects in the augmented reality scene, the method further includes: Receive the second interactive action; In response to the second interactive action, the virtual object is rendered in the augmented reality scene.

14. The method according to claim 13, characterized in that, The step of responding to the second interactive action by rendering the virtual object in the augmented reality scene includes: Based on the second interactive action, the virtual object type is determined, wherein the virtual object type is used to characterize the three-dimensional shape of the virtual object; The virtual object conforming to the virtual object type is rendered in the augmented reality scene.

15. The method according to claim 8, characterized in that, Determining the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera includes: The augmented reality scene is represented using the first coordinate system; Determine the first spatial coordinates of the virtual object in the first coordinate system.

16. The method according to any one of claims 6 to 15, characterized in that, The method further includes: Acquire image data of the first spatial region captured by the first camera, wherein the acquisition time of the image data matches the acquisition time of the point cloud data; The image data is sent to the control processor, wherein the control processor determines the transformation relationship between the first coordinate system and the second coordinate system based on the image data and the point cloud data.

17. A point cloud data segmentation device, characterized in that, include: A receiving module is used to receive first spatial coordinates, wherein the first spatial coordinates are coordinates corresponding to a target spatial region, the target spatial region is a region selected from a predetermined spatial region captured by a first camera, and the first spatial coordinates are coordinates in a first coordinate system corresponding to the first camera. The acquisition module is used to acquire point cloud data obtained by the second camera from shooting the predetermined spatial area; The transformation module is used to transform the first spatial coordinates into second spatial coordinates in the second coordinate system based on the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera. The segmentation module is used to segment the point cloud data according to the second spatial coordinates to obtain target data corresponding to the target spatial region; The receiving module is further configured to receive the first spatial coordinates sent by the augmented reality device. The augmented reality device includes a first camera. The augmented reality device generates an augmented reality scene corresponding to the predetermined spatial area based on image data obtained by the first camera capturing the predetermined spatial area. The first spatial coordinates are coordinates generated by the augmented reality device based on the target spatial area selected in the augmented reality scene. The target spatial area is selected by receiving a first interactive action that interacts with a virtual object in the augmented reality scene. The first interactive action is obtained by: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

18. A point cloud data segmentation device, characterized in that, include: The selection module is used to select a target spatial region within a predetermined spatial region captured by the first camera; The determining module is used to determine the first spatial coordinates of the target spatial region in the first coordinate system corresponding to the first camera; The sending module is used to send the first spatial coordinates to the control processor. The control processor is used to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera. The control processor is also used to segment the point cloud data obtained by the second camera from the predetermined spatial area according to the second spatial coordinates to obtain target data corresponding to the target spatial area. The target space region is selected by receiving a first interactive action that interacts with a virtual object in the augmented reality scene. The first interactive action is obtained by: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

19. A point cloud data segmentation system, characterized in that, include: The augmented reality (AR) device includes a first camera, a control processor, and a second camera; among which, The AR device is configured to generate an augmented reality scene based on image data obtained by the first camera capturing a predetermined spatial area, receive a target spatial area selected in the augmented reality scene, and determine the first spatial coordinates of the target spatial area, wherein the first spatial coordinates are coordinates in the first coordinate system corresponding to the first camera. The AR device is also used to send the first spatial coordinates to the control processor; The second camera is used to capture images of the predetermined spatial area to obtain point cloud data, and is also used to send the point cloud data to the control processor; The control processor is configured to convert the first spatial coordinates into second spatial coordinates in the second coordinate system according to the transformation relationship between the first coordinate system and the second coordinate system corresponding to the second camera, and to segment target data corresponding to the target spatial region from the point cloud data according to the second spatial coordinates; The AR device is further configured to select the target space region by receiving a first interactive action to interact with a virtual object in the AR scene. The first interactive action is obtained by: capturing the action of the entity object and displaying the virtual image and virtual action corresponding to the entity object in the augmented reality scene; determining that the entity object and the virtual object are successfully matched if the virtual image and the virtual object meet the matching rules; and determining the virtual action corresponding to the entity object as the first interactive action.

20. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device where the non-volatile storage medium is located to perform the point cloud data segmentation method according to any one of claims 1 to 16.

21. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store a program, and the processor is used to run the program stored in the memory. When the program is run, it executes the point cloud data segmentation method according to any one of claims 1 to 16.

Images