Virtual-real space personnel mapping positioning system and method

Abstract

The invention relates to the technical field of virtual-real mapping positioning, in particular to a virtual-real space personnel mapping positioning system and method. The system comprises an image acquisition unit, a target detection unit, a camera parameter processing unit, a coordinate conversion unit, a real space model construction unit, a virtual space model construction unit, a coordinate extraction unit, a virtual-real registration unit and a personnel mapping unit. The spatial position of the target person in the real spatial model corresponds to the spatial position of the target person in the virtual spatial model through units in the system. According to the method, accurate positioning and mapping of the target entity personnel in the real space and the virtual space can be realized, so that convenience is provided for various application scenes, and more intelligent and accurate man-machine interaction and service can be realized in the fields of smart home, smart office, virtual reality and the like.

Description

Technical Field

[0001] The present invention relates to the field of virtual-reality mapping and positioning technology, and in particular to a virtual-reality space personnel mapping and positioning system and method. Background Art

[0002] The virtual-real space personnel mapping and positioning system is a positioning system that combines virtual reality technology and actual physical space. It can be used in a variety of application scenarios. The main function of this system is to map people or entities in the virtual space to the actual physical space and perform real-time positioning and tracking.

[0003] Without a virtual-real space personnel mapping and positioning system, it is impossible to accurately locate and map real-world personnel in virtual space, and thus it is impossible to provide more intelligent and precise human-computer interaction and services for intelligent devices and systems, and thus it is impossible to achieve a more realistic and personalized user experience in a virtual reality environment. In addition, in the field of virtual reality, without a virtual-real space personnel mapping and positioning system, it is impossible to provide support for personnel positioning and interaction in a virtual environment, and thus it is impossible to improve the realism and interactivity of the virtual experience. Therefore, the method of virtual-real space personnel mapping and positioning is extremely important. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a virtual-real space personnel mapping and positioning system and method, and the technical solutions adopted are as follows: In a first aspect, an embodiment of the present invention provides a virtual-real space personnel mapping and positioning system, the system comprising the following units: An image acquisition unit, used for acquiring image frames of the target entity person to be processed and the spatial area where the target entity person is located; The target detection unit is used to visually locate the target entity person to be processed and the image frame of the spatial area where the target entity person is located, so as to determine the position coordinates of the spatial area where the target entity person is located in the pixel coordinate system; A camera parameter processing unit is used to obtain point cloud data of a spatial area through a radar, and to obtain camera external parameters by registering the position coordinates of the spatial area and the two-dimensional plane coordinates in the position coordinates of the target entity person and the three-dimensional space coordinates in the point cloud data using a registration algorithm; A coordinate conversion unit, used for converting between two-dimensional plane and three-dimensional space coordinates according to the camera external parameters and the position coordinates of the target entity person, to obtain the converted three-dimensional space coordinates; A real space model building unit, used to build a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determine the spatial position of the target entity object in the real space model; A virtual space model building unit, used to build a virtual space model according to the real space model; A coordinate extraction unit, used to set a plurality of first marking points in the real space model and second marking points corresponding thereto in the virtual space model; and to obtain the real three-dimensional space coordinates of the real space model and the virtual three-dimensional space coordinates of the virtual space model based on the first marking points and the second marking points; A virtual-real registration unit, used to establish a conversion matrix for virtual-real registration by using a space mapping method, and to determine the position correspondence and rotation angle between the real space model and the virtual space model; The personnel mapping unit is used to correspond the spatial position of the target personnel in the real space model with the spatial position of the target personnel in the virtual space model by using the conversion matrix.

[0005] Furthermore, the registration algorithm is:

[0006]

[0007] in, represents the representation function of the first point cloud dataset, represents the centroid of the point cloud, N represents the total number of point clouds, represents the i-th point cloud, t represents the translation vector, and represents the corresponding points of the two point clouds to be registered, R represents the rotation matrix, and K represents the number of corresponding points.

[0008] Furthermore, the coordinate conversion unit further includes: The camera external parameters are received, and based on the camera external parameters, the two-dimensional plane coordinates of the position coordinates of the spatial area and the position coordinates of the target entity person are converted into three-dimensional space coordinates.

[0009] Furthermore, the coordinate extraction unit further includes: A plurality of first marking points are set in the real space model, and a plurality of second marking points are set in the virtual space model corresponding to the plurality of first marking points of the real space model, a plurality of real three-dimensional space coordinates are obtained based on the plurality of first marking points, and a plurality of virtual three-dimensional space coordinates are obtained based on the plurality of second marking points.

[0010] Furthermore, the virtual-real registration unit further includes: According to the real three-dimensional space coordinates and the virtual three-dimensional space coordinates, a transformation matrix for virtual-real registration is established through the space mapping method. The transformation matrix includes a rotation matrix and a translation matrix. The position correspondence and rotation angle between the real space model and the virtual space model are determined according to the transformation matrix.

[0011] Furthermore, the virtual-real registration unit further includes: The transformation matrix is ​​transmitted to the personnel mapping unit, where the spatial mapping method is specifically:

[0012] in, Indicates the coordinate value before transformation. Represents the converted coordinate value, represents the rotation angle, c represents the scaling parameter;

[0013] in, represents the translation value, express , a represents the variable projection representation, Φ represents the matrix representation of the rotation angle, and I represents the full parameter representation matrix.

[0014] Furthermore, the target detection unit also includes the following modules: A person identification module 201 is used to process the spatial region image frame to be processed using an image processing technique to identify a target entity person therein; A personnel positioning module 202 is used to perform visual positioning of the target entity person using image processing technology after identifying the target entity person, so as to determine the position coordinates of the target entity person in a pixel coordinate system; The coordinate calculation module 203 is used to calculate the spatial position coordinates of the target entity person in the virtual scene according to the position coordinates of the target entity person in the pixel coordinate system.

[0015] Furthermore, the coordinate extraction unit also includes the following modules: A first marking point setting module is used to set a number of first marking points in the real space model. The first marking points are known objects or positions, and their position coordinates need to be measured in advance; A second marking point setting module, used to set a plurality of second marking points in the virtual space model corresponding to the plurality of first marking points of the real space model, so as to align the real space model with the virtual space model; The three-dimensional space coordinate extraction module is used to obtain a number of real three-dimensional space coordinates in the real space model based on a number of first marking points, and to obtain a number of virtual three-dimensional space coordinates in the virtual space model based on a number of second marking points.

[0016] Furthermore, the virtual-real registration unit also includes the following modules: A transformation matrix establishment module is used to establish a transformation matrix for virtual-real registration based on real three-dimensional space coordinates and virtual three-dimensional space coordinates using a space mapping method; the transformation matrix includes a rotation matrix and a translation matrix, which are used to position, orient and scale the real space model and the virtual space model; A relationship determination module, used to determine the position correspondence and rotation angle between the real space model and the virtual space model according to the transformation matrix; The conversion matrix transmission module is used to transmit the established conversion matrix to the personnel mapping unit.

[0017] In a second aspect, a method for mapping and positioning personnel in a virtual and real space is provided, the method comprising the following steps: Collecting image frames of the target entity person to be processed and the spatial area where the target entity person is located; Visually locate the target entity person to be processed and the image frame of the space area where the target entity person is located, so as to determine the position coordinates of the space area where the target entity person is located in the pixel coordinate system; The point cloud data of the spatial area is acquired by radar, and the two-dimensional plane coordinates in the position coordinates of the spatial area and the position coordinates of the target entity personnel and the three-dimensional space coordinates in the point cloud data are registered by using a registration algorithm to obtain the camera external parameters; According to the camera external parameters and the position coordinates of the target entity person, conversion is performed between the two-dimensional plane and the three-dimensional space coordinates to obtain the converted three-dimensional space coordinates; Constructing a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determining the spatial position of the target entity object in the real space model; Construct a virtual space model based on the real space model; Setting a plurality of first marking points in the real space model and second marking points corresponding thereto in the virtual space model; acquiring real three-dimensional space coordinates of the real space model and virtual three-dimensional space coordinates of the virtual space model based on the first marking points and the second marking points; The conversion matrix of virtual-real registration is established by using the space mapping method, and the position correspondence and rotation angle between the real space model and the virtual space model are determined; The spatial position of the target person in the real space model is matched with the spatial position of the target person in the virtual space model using a transformation matrix.

[0018] The embodiments of the present invention have at least the following beneficial effects: The present invention can achieve accurate positioning and mapping of target entities in real space and virtual space, thereby providing convenience for various application scenarios. For example, in the fields of smart home, smart office, virtual reality, etc., more intelligent and precise human-computer interaction and services can be achieved, and more efficient and accurate solutions can be provided for security monitoring, personnel tracking and other fields, which helps to improve the level of management and safety control.

[0019] The present invention realizes rapid and accurate acquisition of target entity personnel and spatial areas through an image acquisition unit, providing basic data for subsequent processing and analysis; realizes rapid and accurate positioning of target entity personnel through a target detection unit; realizes processing of camera extrinsic parameters and point cloud data through a camera parameter processing unit, providing basic data for subsequent coordinate conversion and alignment; realizes conversion between different coordinate systems through a coordinate conversion unit, providing basic data for subsequent mapping and positioning; realizes alignment between a real space model and a virtual space model through a virtual-reality alignment unit; realizes marker point matching between the real space model and the virtual space model through a coordinate extraction unit; realizes mapping of target personnel between the real space model and the virtual space model through a personnel mapping unit, providing basic data for subsequent visual effects and interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A system block diagram of a virtual-real space personnel mapping and positioning system provided by an embodiment of the present invention; Figure 2 A schematic diagram of a module of a target detection unit provided by an embodiment of the present invention; Figure 3 A schematic diagram of a module of a virtual-real registration unit provided by an embodiment of the present invention; Figure 4 A schematic diagram of a module of a coordinate extraction unit provided by an embodiment of the present invention; Figure 5 A method flow chart of a method for mapping and positioning personnel in a virtual and real space provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in combination with the accompanying drawings and preferred embodiments, a virtual-real space personnel mapping and positioning system and method proposed by the present invention, its specific implementation method, structure, characteristics and effects are described in detail as follows.

[0023] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0024] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" refers to two or more than two.

[0025] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0026] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0027] The embodiments of the present invention are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0028] The specific scheme of a virtual-real space personnel mapping and positioning system and method provided by the present invention is described in detail below with reference to the accompanying drawings.

[0029] Embodiment 1:

[0030] See also Figure 1 , which shows a system block diagram of a virtual-real space personnel mapping and positioning system provided by an embodiment of the present invention, the system includes the following units: The image acquisition unit 1 is used to acquire image frames of the target entity person to be processed and the space area where the target entity person is located.

[0031] The image acquisition unit acquires images of the target entity person and the spatial area where the target entity person is located through a high-definition camera, and generates image frames of the target entity person to be processed and the spatial area where the target entity person is located.

[0032] The target entity person to be processed and the image frame of the spatial area where the target entity person is located are transmitted to the target detection unit.

[0033] The target detection unit 2 is used to visually locate the target entity person to be processed and the image frame of the spatial area where the target entity person is located, so as to determine the position coordinates of the spatial area where the target entity person is located in the pixel coordinate system.

[0034] The target detection unit receives the image frames of the target entity person to be processed and the space area where the target entity person is located, which are transmitted by the image acquisition unit 1 .

[0035] Through image processing technology, the image frame of the space area where the target entity person is located is visually positioned, so as to determine the position coordinates of the space area where the target entity person is located in the pixel coordinate system, and determine the position coordinates of the target entity person in the pixel coordinate system. That is, the position coordinates of the space area and the position coordinates of the target entity person can be obtained through the target detection unit.

[0036] More specifically, see Figure 2 , which shows a module schematic diagram of a target detection unit provided by an embodiment of the present invention, the unit includes the following modules: A person identification module 201 is used to process the spatial region image frame to be processed using an image processing technique to identify a target entity person therein; A personnel positioning module 202 is used to perform visual positioning of the target entity person using image processing technology after identifying the target entity person, so as to determine the position coordinates of the target entity person in a pixel coordinate system; The coordinate calculation module 203 is used to calculate the spatial position coordinates of the target entity person in the virtual scene according to the position coordinates of the target entity person in the pixel coordinate system, and transmit them to the coordinate conversion unit 3 and the camera parameter processing unit 4.

[0037] The position coordinates of the space area and the position coordinates of the target entity person are transmitted to the coordinate conversion unit 3 and the camera parameter processing unit 4.

[0038] The camera parameter processing unit 4 is used to obtain point cloud data of the spatial area through radar, and to obtain camera external parameters by aligning the two-dimensional plane coordinates in the position coordinates of the spatial area and the position coordinates of the target entity person and the three-dimensional space coordinates in the point cloud data using a registration algorithm.

[0039] The camera parameter processing unit receives the position coordinates of the space area and the position coordinates of the target entity person transmitted by the target detection unit, obtains the point cloud data of the space area through the radar, and uses the registration algorithm to register the two-dimensional plane coordinates in the position coordinates of the space area and the position coordinates of the target entity person and the three-dimensional space coordinates in the point cloud data to obtain the camera external parameters. The camera external parameters are transmitted to the coordinate conversion unit 3.

[0040] The registration algorithm is:

[0041]

[0042] in, represents the representation function of the first point cloud dataset, represents the centroid of the point cloud, N represents the total number of point clouds, represents the i-th point cloud, t represents the translation vector, and represents the corresponding points of the two point clouds to be registered, R represents the rotation matrix, and K represents the number of corresponding points.

[0043] The coordinate conversion unit 3 is used to convert between the two-dimensional plane and the three-dimensional space coordinates according to the camera external parameters and the position coordinates of the target entity person to obtain the converted three-dimensional space coordinates.

[0044] The coordinate conversion unit 3 receives the position coordinates of the spatial area and the position coordinates of the target entity person transmitted by the target detection unit 2 and the camera extrinsic parameters transmitted by the camera parameter processing unit 4, and converts the two-dimensional plane coordinates of the position coordinates of the spatial area and the position coordinates of the target entity person into three-dimensional space coordinates based on the camera extrinsic parameters.

[0045] The converted three-dimensional space coordinates are transmitted to the real space model construction unit 5 .

[0046] The real space model building unit 5 is used to build a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determine the spatial position of the target entity object in the real space model.

[0047] The real space model construction unit 5 receives the three-dimensional space coordinates transmitted by the coordinate conversion unit 3, and constructs a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determines the spatial position of the target entity object in the real space model.

[0048] The constructed real space model is transmitted to the virtual space model construction unit 7 and the coordinate extraction unit 8 respectively, and the spatial position of the target entity object in the real space model is transmitted to the personnel mapping unit 9.

[0049] The virtual space model building unit 7 is used to build a virtual space model according to the real space model.

[0050] The real space model transmitted by the real space model construction unit 5 is received, and a virtual space model is constructed according to the real space model, and the constructed virtual space model is transmitted to the coordinate extraction unit 8 .

[0051] The coordinate extraction unit 8 is used to set a plurality of first marking points in the real space model and corresponding second marking points in the virtual space model; based on the first marking points and the second marking points, the real three-dimensional space coordinates of the real space model and the virtual three-dimensional space coordinates of the virtual space model are obtained.

[0052] The coordinate extracting unit 8 receives the real space model transmitted from the real space model constructing unit 5 and the virtual space model transmitted from the virtual space model constructing unit 7 .

[0053] A plurality of first marking points are set in the real space model, and a plurality of second marking points are set in the virtual space model corresponding to the plurality of first marking points of the real space model, a plurality of real three-dimensional space coordinates are obtained based on the plurality of first marking points, and a plurality of virtual three-dimensional space coordinates are obtained based on the plurality of second marking points.

[0054] The plurality of real three-dimensional space coordinates and the plurality of virtual three-dimensional space coordinates are transmitted to the virtual-real registration unit 6 .

[0055] More specifically, see Figure 4 , which shows a schematic diagram of a module of a coordinate extraction unit provided by an embodiment of the present invention, the unit includes the following modules: The first marking point setting module 801 is used to set a plurality of first marking points in the real space model. The first marking points are known objects or positions, and their position coordinates need to be measured in advance. A second marking point setting module 802 is used to set a plurality of second marking points in the virtual space model corresponding to a plurality of first marking points of the real space model, so as to align the real space model with the virtual space model; The three-dimensional space coordinate extraction module 803 is used to obtain a number of real three-dimensional space coordinates in the real space model based on a number of first marking points, and to obtain a number of virtual three-dimensional space coordinates in the virtual space model based on a number of second marking points.

[0056] The virtual-real registration unit 6 is used to establish a conversion matrix for virtual-real registration by using a space mapping method, and to determine the position correspondence and rotation angle between the real space model and the virtual space model.

[0057] The virtual-real registration unit 6 receives a plurality of real three-dimensional space coordinates and a plurality of virtual three-dimensional space coordinates transmitted by the coordinate extraction unit 8 .

[0058] According to the real three-dimensional space coordinates and the virtual three-dimensional space coordinates, a transformation matrix for virtual-real registration is established through the space mapping method. The transformation matrix includes a rotation matrix and a translation matrix. The position correspondence and rotation angle between the real space model and the virtual space model are determined according to the transformation matrix.

[0059] The conversion matrix is ​​transmitted to the personnel mapping unit 9, wherein the spatial mapping method is specifically:

[0060] in, Indicates the coordinate value before transformation. Represents the converted coordinate value, represents the rotation angle, c represents the scaling parameter;

[0061] in, represents the translation value, express , a represents the variable projection representation, Φ represents the matrix representation of the rotation angle, and I represents the full parameter representation matrix; More specifically, see Figure 3 , which shows a module schematic diagram of a virtual-real registration unit provided by an embodiment of the present invention, the unit includes the following modules: The conversion matrix establishment module 601 is used to establish a conversion matrix for virtual-real registration based on the real three-dimensional space coordinates and the virtual three-dimensional space coordinates using a space mapping method; the conversion matrix includes a rotation matrix and a translation matrix, which are used to position, orient and scale the real space model and the virtual space model; A relationship determination module 602 is used to determine the position correspondence and rotation angle between the real space model and the virtual space model according to the transformation matrix; The conversion matrix transmission module 603 is used to transmit the established conversion matrix to the personnel mapping unit 9 .

[0062] The personnel mapping unit 9 is used to correspond the spatial position of the target personnel in the real space model with the spatial position of the target personnel in the virtual space model by using the conversion matrix.

[0063] The personnel mapping unit 9 receives the conversion matrix transmitted by the virtual-real registration unit 6 and the spatial position of the target entity object in the real space model transmitted by the real space model construction unit 5 .

[0064] The spatial position of the target person in the real space model is matched with the spatial position of the target person in the virtual space model by using the conversion matrix.

[0065] Embodiment 2:

[0066] See also Figure 5 , which shows a method flow chart of a virtual-real space personnel mapping and positioning method provided by an embodiment of the present invention, a virtual-real space personnel mapping and positioning method is applicable to a virtual-real space personnel mapping and positioning system, comprising the following steps: S1, collecting image frames of the target entity person to be processed and the spatial area where the target entity person is located.

[0067] Step S1 is used for image acquisition. The spatial region image frame is acquired by a high-definition camera, and the spatial region image frame is also the image frame to be processed.

[0068] S2, visually positioning the target entity person to be processed and the image frame of the spatial area where the target entity person is located, so as to determine the position coordinates of the spatial area where the target entity person is located in the pixel coordinate system.

[0069] Step S2 is used for target detection. After receiving and processing the spatial region image frame, the target entity person is identified and located using image processing technology to determine the position coordinates of the target entity person in the pixel coordinate system.

[0070] S3, acquiring point cloud data of the spatial area through radar, and registering the position coordinates of the spatial area and the two-dimensional plane coordinates in the position coordinates of the target entity person and the three-dimensional space coordinates in the point cloud data using a registration algorithm to obtain camera external parameters.

[0071] Step S3 is used for camera parameter processing. After receiving the position coordinates of the target entity person and the position coordinates of the spatial area, the point cloud data of the spatial area is obtained to obtain the camera external parameters.

[0072] S4, converting between the two-dimensional plane and the three-dimensional space coordinates according to the camera external parameters and the position coordinates of the target entity person to obtain the converted three-dimensional space coordinates.

[0073] Step S4 is used for coordinate conversion. After receiving the camera external parameters and the position coordinates of the target entity person, the converted three-dimensional space coordinates are obtained by analysis.

[0074] S5, constructing a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determining the spatial position of the target entity object in the real space model.

[0075] Step S5 is used for constructing a real space model. After receiving the three-dimensional space coordinates, the spatial position of the target entity object in the real space model is analyzed and obtained.

[0076] S6, constructing a virtual space model based on the real space model.

[0077] Step S6 is used for constructing a virtual space model. After receiving the real space model, the virtual space model is constructed.

[0078] S7, setting a plurality of first marking points in the real space model and corresponding second marking points in the virtual space model; acquiring the real three-dimensional space coordinates of the real space model and the virtual three-dimensional space coordinates of the virtual space model based on the first marking points and the second marking points.

[0079] Step S7 is for coordinate extraction. After receiving the real space model and the virtual space model, the real three-dimensional space coordinates of the real space model and the virtual three-dimensional space coordinates of the virtual space model are determined.

[0080] S8, using the space mapping method to establish a conversion matrix for virtual-real registration, and determine the position correspondence and rotation angle between the real space model and the virtual space model.

[0081] Step S8 is for virtual-real registration. After receiving the real three-dimensional space coordinates and the virtual three-dimensional space coordinates, the position correspondence and the rotation angle between the real space model and the virtual space model are determined.

[0082] S9, using a conversion matrix to match the spatial position of the target person in the real space model with the spatial position of the target person in the virtual space model.

[0083] Step S9 is for personnel mapping. After receiving the conversion matrix and the spatial position of the target entity object in the real space model, the spatial position of the target person in the real space model is matched with the spatial position of the target person in the virtual space model.

[0084] It should be noted that: the system provided in the above embodiment is only illustrated by the division of the above functional units / modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the shopping guide device and the image extension method embodiment provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0085] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0086] It should be noted that the sequence of the above embodiments of the present invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0088] The above contents are only specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A virtual-real space personnel mapping and positioning system, characterized in that: The system includes the following units: An image acquisition unit, used for acquiring image frames of the target entity person to be processed and the spatial area where the target entity person is located; The target detection unit is used to visually locate the target entity person to be processed and the image frame of the spatial area where the target entity person is located, so as to determine the position coordinates of the spatial area where the target entity person is located in the pixel coordinate system; A camera parameter processing unit is used to obtain point cloud data of a spatial area through a radar, and to obtain camera extrinsics by registering the two-dimensional plane coordinates in the position coordinates of the spatial area and the position coordinates of the target entity person and the three-dimensional space coordinates in the point cloud data using a registration algorithm; A coordinate conversion unit, used for converting between two-dimensional plane and three-dimensional space coordinates according to the camera external parameters and the position coordinates of the target entity person, to obtain the converted three-dimensional space coordinates; A real space model building unit, used to build a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determine the spatial position of the target entity object in the real space model; A virtual space model building unit, used to build a virtual space model according to the real space model; A coordinate extraction unit, used to set a plurality of first marking points in the real space model and second marking points corresponding thereto in the virtual space model; and to obtain the real three-dimensional space coordinates of the real space model and the virtual three-dimensional space coordinates of the virtual space model based on the first marking points and the second marking points; A virtual-real registration unit, used to establish a conversion matrix for virtual-real registration by using a space mapping method, and to determine the position correspondence and rotation angle between the real space model and the virtual space model; The personnel mapping unit is used to correspond the spatial position of the target personnel in the real space model with the spatial position of the target personnel in the virtual space model by using the conversion matrix.

2. A virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The registration algorithm is: in, represents the representation function of the first point cloud dataset, represents the centroid of the point cloud, N represents the total number of point clouds, represents the i-th point cloud, t represents the translation vector, and represents the corresponding points of the two point clouds to be registered, R represents the rotation matrix, and K represents the number of corresponding points.

3. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The coordinate conversion unit also includes: The camera external parameters are received, and based on the camera external parameters, the two-dimensional plane coordinates of the position coordinates of the spatial area and the position coordinates of the target entity person are converted into three-dimensional space coordinates.

4. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The coordinate extraction unit further includes: A plurality of first marking points are set in the real space model, and a plurality of second marking points are set in the virtual space model corresponding to the plurality of first marking points of the real space model, a plurality of real three-dimensional space coordinates are obtained based on the plurality of first marking points, and a plurality of virtual three-dimensional space coordinates are obtained based on the plurality of second marking points.

5. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The virtual-real registration unit also includes: According to the real three-dimensional space coordinates and the virtual three-dimensional space coordinates, a transformation matrix for virtual-real registration is established through the space mapping method. The transformation matrix includes a rotation matrix and a translation matrix. The position correspondence and rotation angle between the real space model and the virtual space model are determined according to the transformation matrix.

6. The virtual-real space personnel mapping and positioning system according to claim 5, characterized in that: The virtual-real registration unit also includes: The transformation matrix is ​​transmitted to the personnel mapping unit, where the spatial mapping method is specifically: in, Indicates the coordinate value before transformation. Represents the converted coordinate value, represents the rotation angle, c represents the scaling parameter; in, represents the translation value, express , a represents the variable projection representation, Φ represents the matrix representation of the rotation angle, and I represents the full parameter representation matrix.

7. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The target detection unit also includes the following modules: A personnel recognition module is used to process the image frame of the spatial region to be processed using an image processing technology to recognize a target entity person therein; A personnel positioning module is used to visually locate the target entity person using image processing technology after identifying the target entity person, so as to determine the position coordinates of the target entity person in a pixel coordinate system; The coordinate calculation module is used to calculate the spatial position coordinates of the target entity person in the virtual scene according to the position coordinates of the target entity person in the pixel coordinate system.

8. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The coordinate extraction unit also includes the following modules: A first marking point setting module is used to set a number of first marking points in the real space model. The first marking points are known objects or positions, and their position coordinates need to be measured in advance; A second marking point setting module, used to set a plurality of second marking points in the virtual space model corresponding to the plurality of first marking points of the real space model, so as to align the real space model with the virtual space model; The three-dimensional space coordinate extraction module is used to obtain a number of real three-dimensional space coordinates in the real space model based on a number of first marking points, and to obtain a number of virtual three-dimensional space coordinates in the virtual space model based on a number of second marking points.

9. The virtual-real space personnel mapping and positioning system according to claim 1, characterized in that: The virtual-real registration unit also includes the following modules: A transformation matrix establishment module is used to establish a transformation matrix for virtual-real registration based on real three-dimensional space coordinates and virtual three-dimensional space coordinates using a space mapping method; the transformation matrix includes a rotation matrix and a translation matrix, which are used to position, orient and scale the real space model and the virtual space model; A relationship determination module, used to determine the position correspondence and rotation angle between the real space model and the virtual space model according to the transformation matrix; The conversion matrix transmission module is used to transmit the established conversion matrix to the personnel mapping unit.

10. A method for mapping and positioning personnel in virtual and real space, characterized in that: The method comprises the following steps: Collecting image frames of the target entity person to be processed and the spatial area where the target entity person is located; Visually locate the target entity person to be processed and the image frame of the space area where the target entity person is located, so as to determine the position coordinates of the space area where the target entity person is located in the pixel coordinate system; The point cloud data of the spatial area is acquired by radar, and the two-dimensional plane coordinates in the position coordinates of the spatial area and the position coordinates of the target entity personnel and the three-dimensional space coordinates in the point cloud data are registered by using a registration algorithm to obtain the camera external parameters; According to the camera external parameters and the position coordinates of the target entity person, conversion is performed between the two-dimensional plane and the three-dimensional space coordinates to obtain the converted three-dimensional space coordinates; Constructing a real space model of the space plane where the target entity object is located according to the three-dimensional space coordinates, and determining the spatial position of the target entity object in the real space model; Construct a virtual space model based on the real space model; Setting a plurality of first marking points in the real space model and second marking points corresponding thereto in the virtual space model; acquiring real three-dimensional space coordinates of the real space model and virtual three-dimensional space coordinates of the virtual space model based on the first marking points and the second marking points; The conversion matrix of virtual-real registration is established by using the space mapping method, and the position correspondence and rotation angle between the real space model and the virtual space model are determined; The spatial position of the target person in the real space model is matched with the spatial position of the target person in the virtual space model using a transformation matrix.

Images