An image processing method, apparatus and device

Abstract

The embodiments of the present application relate to the field of computers, and disclose an image processing method, apparatus and device. Obtain at least two pieces of image information of a target scene; respectively obtain pictures of at least three objects from each of the at least two pieces of image information; for each object, set a calibration point in each of the at least two pieces of image information to obtain the spatial coordinates of each calibration point corresponding to each piece of image information; for each object, obtain the relationship features between the at least two pieces of image information; convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image to obtain the target image. It can be seen that this technical solution establishes the relationship features between the image information obtained by multiple devices, and realizes the point cloud merging of the object coordinates corresponding to multiple devices based on these features. In this way, the accuracy of the final image acquisition by multiple image acquisition devices is ensured, and the placement flexibility of the image acquisition devices is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computers, and relate to an image processing method, apparatus, and device. Background Art

[0002] A body-sensing game console relies on video motion capture technology to timely reflect human body movements into the game system and interact with the game scene through changes in human body movements. A body-sensing game console usually includes a host and a control terminal. The host is responsible for operation and new model processing. The control terminal is essentially a three-dimensional (3D) image acquisition device, which is responsible for acquiring dynamic information. Different from a conventional two-dimensional (2D) image acquisition device, a 3D image acquisition device can not only capture the color information of a scene, but also obtain the depth information of the scene, providing a more complete spatial position description of the captured object for the host.

[0003] In the same game scene, in order to provide a more accurate spatial position description of the captured object in the scene, multiple 3D image acquisition devices are usually deployed to obtain the scene information of the captured object. However, since there are deviations in the spatial positions of each 3D image acquisition device relative to the captured object, the depth information of the captured object obtained by each 3D image acquisition device is different, so that the spatial position of the captured object in the finally output image is also distorted. For the problem of distorted output images existing in multiple 3D image acquisition devices at the present stage, the solution is usually to fix the positions of each 3D image acquisition device in the scene, and offset and correct the coordinates of the captured object in the images obtained by each device through the fixed spatial distance between them, and finally output the corrected image at the output end.

[0004] However, the method of fixing the positions of 3D image acquisition devices limits the actual use scenarios of body-sensing game console products. And in the use scenarios where the positions of 3D image acquisition devices are fixed, it increases the difficulty for users to perform high-precision image calibration. Summary of the Invention

[0005] Embodiments of the present application provide an image method, apparatus, and device to solve the problem of distorted images captured by existing multiple 3D acquisition devices.

[0006] In a first aspect, embodiments of the present application provide an image processing method, the method including:

[0007] Obtain at least two image information of a target scene, where the at least two image information are respectively acquired from different angles by at least two pre-deployed image acquisition devices;

[0008] Obtain pictures of at least three objects from each of the at least two pieces of image information, where each of the at least three objects is included in the at least two pieces of image information;

[0009] For each of the at least three objects, set a calibration point in the at least two pieces of image information respectively, and obtain the spatial coordinates of each calibration point corresponding to each piece of image information;

[0010] For each of the at least three objects, based on the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information, obtain the relationship characteristics between the at least two pieces of image information;

[0011] Based on the relationship characteristics between each piece of image information and the reference image, convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image to obtain the target image, where the reference image is any one of the at least two pieces of image information.

[0012] In some possible implementation manners, the image information includes color information and depth information.

[0013] In some possible implementation manners, for the pictures of the at least three objects, the pictures of all the objects are from the image information of the same image frame.

[0014] In some possible implementation manners, before obtaining the pictures of the at least three objects, the image processing method further includes establishing a corresponding relationship between the color information and the depth information according to a preset correction matrix.

[0015] In some possible implementation manners, the relationship characteristics between the two pieces of image information include: the offset matrix between the two pieces of image information.

[0016] In some possible implementation manners, the image processing method further includes: after obtaining the target image, judging the target image according to a preset rule and obtaining a corresponding result value. If the result value corresponding to the target image is within a preset threshold range, it is output as the final result;

[0017] If the result value corresponding to the target image is not within the preset threshold range, re-obtain pictures of at least three new objects in the at least two pieces of image information, where each of the at least three new objects is included in the at least two pieces of image information and is different from the at least three objects corresponding to the image frame of the target image. In a second aspect, an embodiment of the present application further provides an image processing apparatus, and the apparatus includes:

[0018] A first acquisition module, configured to acquire at least two image information of a target scene, where the at least two image information are respectively acquired by at least two pre-deployed image acquisition devices from different angles;

[0019] A second acquisition module, configured to respectively acquire pictures of at least three objects from each of the at least two image information, where each of the at least three objects is included in the at least two image information;

[0020] A calibration module, configured to set a calibration point in each of the at least two image information for each of the at least three objects, and obtain the spatial coordinates of each calibration point corresponding to each image information;

[0021] A first processing module, configured to obtain the relationship features between at least two image information for each of the at least three objects according to the spatial coordinates of the calibration points corresponding to each image information and the at least two image information;

[0022] A second processing module, configured to convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image according to the relationship features between each image information and the reference image, to obtain a target image, where the reference image is any one of the at least two image information.

[0023] In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method in the first aspect or any possible implementation manner of the first aspect by executing the computer instructions.

[0024] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0025] The embodiment of the present application provides a technical solution for an image processing method. In this solution, first, at least two image acquisition devices preset at different angles respectively acquire image information of a scene. Then, pictures of at least three objects that appear in at least two pieces of acquired image information are respectively selected from the at least two pieces of image information. For each object among the at least three objects, a calibration point is respectively set in the at least two pieces of image information to obtain the spatial coordinates of each calibration point corresponding to each piece of image information. And based on the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information, the relationship features between the at least two pieces of image information are obtained. Finally, based on the relationship features between each piece of image information and a reference image, the spatial coordinates of each object in the image information are converted to the spatial coordinates of the corresponding object in the reference image to obtain a target image. It can be seen that this technical solution establishes the relationship features between image information obtained by multiple devices and realizes the point cloud merging of object coordinates corresponding to multiple devices based on these features. In this way, the accuracy of the final image acquisition by multiple image acquisition devices is ensured and the placement flexibility of the image acquisition devices is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic flowchart of the image processing method provided by the embodiment of the present application;

[0027] Figure 2a is a schematic diagram of the selection of calibration points of device K1 provided by the embodiment of the present application;

[0028] Figure 2b is a schematic diagram of the selection of calibration points of device K2 provided by the embodiment of the present application;

[0029] Figure 3 is a schematic diagram of an exemplary composition of the image processing device provided by the embodiment of the present application;

[0030] Figure 4 is a schematic diagram of an exemplary structure of the image processing device provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The terms used in the following embodiments of the present application are for the purpose of describing alternative embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above", "said", "this" are also intended to include the plural forms. It should also be understood that although the terms first, second, etc. may be used in the following embodiments to describe a certain type of object, the object is not limited to these terms. These terms are used to distinguish specific objects of this type of object. For example, the same applies to other types of objects that may be described by the terms first, second, etc. in the following embodiments, and will not be elaborated here.

[0032] An embodiment of the present application provides a technical solution for an image processing method. In this solution, first, at least two image acquisition devices preset at different angles respectively acquire image information of a scene. Then, pictures of at least three objects that appear in all of the at least two acquired image information are respectively selected from the at least two image information. For each object among the at least three objects, a calibration point is set in each of the at least two image information to obtain the spatial coordinates of each calibration point corresponding to each image information. And based on the spatial coordinates of the calibration points corresponding to each image information and the at least two image information, a relationship feature between the at least two image information is obtained. Finally, based on the relationship feature between each image information and a reference image, the spatial coordinates of each object in the image information are converted to the spatial coordinates of the corresponding object in the reference image to obtain a target image. It can be seen that this technical solution establishes a relationship feature between image information obtained by multiple devices and realizes point cloud merging of object coordinates corresponding to multiple devices with this feature. In this way, the accuracy of the final image acquisition by multiple image acquisition devices is ensured and the placement flexibility of the image acquisition devices is improved.

[0033] Any electronic device involved in an embodiment of the present application can be an electronic device such as a mobile phone, a tablet computer, a wearable device (such as a smart watch, a smart bracelet, etc.), a notebook computer, a desktop computer, and a vehicle-mounted device. The electronic device is pre-installed with a software deployment application program. It can be understood that the specific type of the electronic device is not limited in any way in the embodiment of the present application.

[0034] A body-sensing game console relies on video motion capture technology to timely reflect human movements into the game system and interact with the game scene through changes in human movements. A body-sensing game console usually includes a host and a control end. The host is responsible for running and processing new models, and the control end is essentially a three-dimensional (3D) image acquisition device responsible for collecting dynamic information. Different from a conventional two-dimensional (2D) image acquisition device, a 3D image acquisition device can not only capture the color information of a scene but also obtain the depth information of the scene, providing a more complete spatial position description of the captured object for the host.

[0035] In the same game scenario, in order to more accurately describe the spatial position of the captured object in the scenario, multiple 3D image acquisition devices are usually deployed to obtain the scenario information of the captured object. However, since there are deviations in the spatial positions of each 3D image acquisition device relative to the captured object, the depth information of the captured object obtained by each 3D image acquisition device is different. As a result, the spatial position of the captured object in the finally output image is also distorted. For the problem of distorted output images existing in multiple 3D image acquisition devices at the present stage, the solution is usually to fix the positions of each 3D image acquisition device in the scenario, and correct the offset of the coordinates of the captured object in the images obtained by each device through the fixed spatial distance between them, and finally output the corrected image at the output end.

[0036] However, the method of fixing the positions of 3D image acquisition devices limits the actual usage scenarios of the somatosensory game console products. And in the usage scenarios where the positions of 3D image acquisition devices are fixed, it increases the difficulty for users to perform high-precision image calibration.

[0037] The following is a description of several exemplary embodiments, explaining the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application.

[0038] In the first aspect of the present application, an image processing method is proposed. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the image processing method provided by the embodiments of the present application, including the following steps:

[0039] Obtain at least two pieces of image information of the target scenario, where the at least two pieces of image information are respectively acquired from different angles by at least two pre-deployed image acquisition devices;

[0040] Respectively obtain pictures of at least three objects from each of the at least two pieces of image information, and each of the at least three objects is included in the at least two pieces of image information;

[0041] For each object among the at least three objects, set a calibration point in each of the at least two pieces of image information to obtain the spatial coordinates of each calibration point corresponding to each piece of image information;

[0042] For each object among the at least three objects, based on the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information, obtain the relationship features between the at least two pieces of image information;

[0043] According to the relationship characteristics between each piece of image information and the reference image, convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image to obtain the target image, where the reference image is any one of the at least two pieces of image information.

[0044] Optionally, the image information includes color information and depth information.

[0045] Exemplarily, taking the somatosensory game console K device as an example (here, the K device is only used to refer to the device type and is not limited), at least two K devices (equivalent to the above-mentioned image acquisition devices) are deployed in advance, and the deployed at least two K devices are connected to the same host so that the host presents the final image (equivalent to the above-mentioned target image). The devices can be placed at any angle relative to each other, enabling the host to obtain image information from different angles in the same scene. The image information includes color information and depth information, where the depth information can generally be understood as the three-dimensional coordinate values of each point of the detected object.

[0046] Generally, for an object in the same scene, due to different shooting angles, the color information of the object (such as the reflectivity of the object at the current shooting angle and the brightness and darkness at the current shooting angle, etc.) and the depth information (such as the phenomenon of "objects appear larger when closer and smaller when farther away") will vary with different shooting angles.

[0047] Optionally, to reduce this difference, a distortion correction matrix (i.e., the above-mentioned correction matrix) is usually used to establish a pairwise correspondence between color and depth, and denoising and screening are performed on abnormal depth information content in the image information (for example: removing data with abnormal depth information or data with missing depth information). This can reduce interference factors in the subsequent point cloud merging process and improve the accuracy of image processing.

[0048] Exemplarily, the situations of shooting by devices with different placement angles in the same scene are as follows. Figure 2a It is a schematic diagram of the scene at the angle of the K1 device. Figure 2bIt is a schematic diagram of the scene from the perspective of device K2. Obviously, the image information captured by devices K1 and K2 has different angles. Among the image information corresponding to devices K1 and K2 (where K1 and K2 only refer to device numbers and are not limited), three objects A, B, and C (where A, B, and C only refer to objects and are not limited) that appear in the scene are selected, and A, B, and C are simultaneously captured in the respective corresponding image information by devices K1 and K2 in the same image frame. The three objects that appear in the scene can also be completely static objects (i.e., objects whose spatial position information cannot be changed by any influence, such as fixed items placed in the scene, etc.). When the object is a completely static object, the acquisition of the object image will not be limited to the same frame of image. Taking A, B, and C as calibration points, then in the image information obtained by devices K1 and K2 respectively, points A, B, and C form two triangles respectively. The triangle formed in the image information corresponding to device K1 is denoted as ΔA0B0C0, and the triangle formed in the image information corresponding to device K2 is denoted as ΔA1B1C1. In this way, according to the coordinate systems preset by devices K1 and K2 respectively, the spatial coordinates of each vertex of the triangle in their respective image information (i.e., A0, B0, C0, A1, B1, and C1) can be obtained. And according to the coordinate differences of each vertex in the two image information (the difference between A0 and A1, the difference between B0 and B1, and the difference between C0 and C1), the offset rotation between the two image information (i.e., the above relationship feature) can be obtained. The offset matrix is denoted as M (equivalent to setting a calibration point for each object among the at least three objects in each of the at least two image information respectively, obtaining the spatial coordinates of each calibration point corresponding to each image information, and obtaining the relationship feature between the at least two image information based on the spatial coordinates of the calibration points corresponding to each image information for each object among the at least three objects and the at least two image information).

[0049] Optionally, according to the offset matrix M, ΔA1B1C1 is offset to the image information corresponding to device K1 to achieve point cloud merging. The calculation process of the offset matrix M specifically includes:

[0050] According to the vector and Represent the rotation from the former to the latter with the quaternion q1;

[0051] Let a perpendicular line from point C to side AB with the foot of the perpendicular being P. Respectively calculate the vectors according to the coordinates of each triangle

[0052] The vector obtained after transformation by q1 is Calculate the result from to The rotation is denoted as the quaternion q2;

[0053] The final rotation coefficient r is obtained by multiplying q1 by q2. After rotation by r, the two triangles are completely parallel (using side AB and the altitude on side AB to determine the two rotations can ensure that the second rotation does not affect side AB that has been aligned after the first rotation);

[0054] Left multiplying r by A0 gives A'0, and the displacement vector is denoted as t;

[0055] Based on the displacement vector t, the rotation coefficient r, and the scaling coefficient s, the offset matrix M (i.e., the above-mentioned relationship feature) between the image information corresponding to device K1 and the image information corresponding to device K2 can be obtained.

[0056] Optionally, when there are more than 2 image processing devices, the above process can be repeated for each device to obtain the offset matrix (i.e., the above-mentioned relationship feature) between every two image devices.

[0057] Exemplarily, after obtaining the offset matrix M, taking the space of the image information acquired by device K1 as the standard, the coordinates of the point cloud data in the image information collected by device K2 are converted into the space coordinates corresponding to K1, realizing point cloud merging (i.e., according to the relationship feature between each image information and the reference image, converting the space coordinates of each object in the image information to the space coordinates of the corresponding object in the reference image to obtain the target image, and the reference image is any one of the at least two image information).

[0058] Optionally, an image processor (Graphics Processing Unit, GPU) tool can be used to achieve real-time merging, improving the efficiency of point cloud merging.

[0059] Optionally, the above operations can be repeated for sides BC and AC, and after obtaining the respective corresponding offset matrices, mean processing is performed to reduce single-point errors.

[0060] Exemplarily, the method further includes a pre-calibration action. The scenarios where the pre-calibration action occurs include: in a scenario, at least two image acquisition devices cannot fully capture the entire scene, for example, missing the necessary features of the "ground" in the scene. Therefore, it is necessary to perform a pre-calibration action on the image acquisition devices that cannot capture the entire scene. The specific method includes:

[0061] Within the common capture range of at least two image acquisition devices that need to be pre-calibrated, a calibration template is set (for example, if two image acquisition devices that need to be pre-calibrated can both capture the wall "W" set in the east of the scene, then the wall "W" can be used as the common capture range, and a calibration template is set within this range);

[0062] Obtain calibration template information, and perform the above image processing method (not elaborated here one by one) on at least two image acquisition devices to be pre-calibrated according to the calibration template, obtain the final image, and achieve point cloud merging. In this way, through point cloud coordinate merging and normalization processing, pre-calibration of the image acquisition device can be achieved.

[0063] Optionally, the calibration template includes at least three calibration points, and the calibration points are used to provide calibration coordinate information for each of at least two image acquisition devices to be pre-calibrated.

[0064] Optionally, the image processing method further includes: after obtaining the target image, judging the target image according to a preset rule, and obtaining a corresponding result value. If the result value corresponding to the target image is within a preset threshold range, it is output as the final result;

[0065] If the result value corresponding to the target image is not within the preset threshold range, pictures of at least three new objects are re-obtained from at least two pieces of image information. Each object in the at least three new objects is included in the at least two pieces of image information and is different from at least three objects corresponding to the image frame of the target image.

[0066] The embodiment of the present application provides a technical solution for an image processing method. In this solution, first, image information of a scene is respectively obtained by at least two image acquisition devices preset at different angles, and then pictures of at least three objects that appear in at least two pieces of image information are respectively selected from the at least two pieces of image information obtained. For each object in the at least three objects, a calibration point is set in each of the at least two pieces of image information to obtain the spatial coordinates of each calibration point corresponding to each piece of image information, and according to the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information, the relationship characteristics between the at least two pieces of image information are obtained. Finally, according to the relationship characteristics between each piece of image information and the reference image, the spatial coordinates of each object in the image information are converted to the spatial coordinates of the corresponding object in the reference image to obtain the target image. It can be seen that this technical solution establishes the relationship characteristics between image information obtained by multiple devices and realizes the point cloud merging of object coordinates corresponding to multiple devices through this characteristic. In this way, the accuracy of the final image acquisition by multiple image acquisition devices is ensured and the placement flexibility of the image acquisition device is improved.

[0067] The above embodiments have introduced various implementation manners of the image processing method provided in the embodiments of the present application from aspects such as obtaining at least one image information of a scene, selecting an object picture, setting a calibration point, establishing relationship features between image information, and coordinate conversion. It should be understood that for the processing steps such as obtaining at least one image information of a scene, selecting an object picture, setting a calibration point, establishing relationship features between image information, and coordinate conversion, the embodiments of the present application can implement the above functions in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the manner of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0068] For example, if the above implementation steps are implemented by software modules to implement corresponding functions. As Figure 3 shown, the image processing device may include a first acquisition module, a second acquisition module, a calibration module, a first processing module, and a second processing module. The image processing device can be used to perform some or all of the operations of the above image processing method.

[0069] For example:

[0070] The first acquisition module is used to acquire at least two image information of a target scene, and the at least two image information are respectively acquired from different angles by at least two pre-deployed image acquisition devices;

[0071] The second acquisition module is used to respectively acquire pictures of at least three objects from each of the at least two image information, and each of the at least three objects is included in the at least two image information;

[0072] The calibration module is used to set a calibration point for each of the at least three objects in the at least two image information respectively, and obtain the spatial coordinates of each calibration point corresponding to each image information;

[0073] The first processing module is used to, for each of the at least three objects, obtain relationship features between at least two image information according to the spatial coordinates of the calibration points corresponding to each image information and the at least two image information;

[0074] The second processing module is used to convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image according to the relationship features between each image information and the reference image, and obtain a target image, where the reference image is any one of the at least two image information.

[0075] An embodiment of the present application provides a technical solution for an image processing method. In this solution, first, at least two image acquisition devices preset at different angles respectively acquire image information of a scene. Then, pictures of at least three objects that appear in all of the at least two acquired image information are respectively selected from the at least two acquired image information. For each object among the at least three objects, a calibration point is respectively set in the at least two image information to obtain the spatial coordinates of each calibration point corresponding to each image information. And based on the spatial coordinates of the calibration points corresponding to each image information and the at least two image information, relationship features between the at least two image information are obtained. Finally, based on the relationship features between each image information and a reference image, the spatial coordinates of each object in the image information are converted to the spatial coordinates of the corresponding object in the reference image to obtain a target image. It can be seen that this technical solution establishes relationship features between image information obtained by multiple devices and realizes point cloud merging of object coordinates corresponding to multiple devices with these features. In this way, the accuracy of the final image acquisition by multiple image acquisition devices is ensured and the placement flexibility of the image acquisition devices is improved.

[0076] It can be understood that the functions of the above-mentioned various modules can be integrated into a hardware entity for implementation. For example, the acquisition module can be integrated into the transceiver for implementation, and the generation module, selection module, and construction module can be integrated in the processor for implementation. The programs and instructions for implementing the functions of the above-mentioned various modules can be maintained in the memory. As Figure 4 shown, an electronic device is provided. The electronic device includes a processor, a transceiver, and a memory. The transceiver is used to execute the acquisition of image information and object pictures in the image generation method. The memory is used to store the programs / codes pre-installed in the foregoing deployment device, and can also store the codes for the processor to execute. When the processor runs the codes stored in the memory, the electronic device is caused to execute some or all of the operations of the software deployment method in the above-mentioned method.

[0077] For the specific process, refer to the embodiments of the above-mentioned method, and details are not described here again.

[0078] In specific implementation, corresponding to the foregoing electronic device, an embodiment of the present application further provides a computer storage medium. Among them, the computer storage medium provided in the electronic device can store a program. When the program is executed, it can implement some or all of the steps in the embodiments of the above-mentioned software deployment method. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0079] One or more of the above modules or units may be implemented in software, hardware, or a combination of both. When any of the above modules or units is implemented in software, the software exists in the form of computer program instructions and is stored in a memory. The processor may be used to execute the program instructions and implement the above method flow. The processor may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or various computing devices that run software such as an artificial intelligence processor. Each computing device may include one or more cores for executing software instructions for arithmetic operations or processing. The processor may be built into a system-on-chip (SoC) or an application specific integrated circuit (ASIC), or it may be an independent semiconductor chip. In addition to the cores in the processor for executing software instructions for arithmetic operations or processing, it may further include necessary hardware accelerators, such as a field programmable gate array (FPGA), a programmable logic device (PLD), or a logic circuit for implementing dedicated logic operations.

[0080] When the above modules or units are implemented in hardware, the hardware may be any one or any combination of a CPU, a microprocessor, a DSP, an MCU, an artificial intelligence processor, an ASIC, an SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator, or a non-integrated discrete device, which may run the necessary software or execute the above method flow without relying on software.

[0081] Furthermore, Figure 4 It may further include a bus interface. The bus interface may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by the processor and a memory represented by the memory. The bus interface may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described herein. The bus interface provides an interface. The transceiver provides a unit for communicating with various other devices on a transmission medium. The processor is responsible for managing the bus architecture and general processing, and the memory may store data used by the processor when performing operations.

[0082] When the above modules or units are implemented in software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0083] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the processes do not imply the order of execution, and the order of execution of the processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation processes of the embodiments.

[0084] Each part of this specification is described in a progressive manner. For the same or similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the description in the method embodiment part for the relevant parts.

[0085] Although the alternative embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0086] The above-described specific embodiments further elaborate on the purpose, technical solution, and beneficial effects of the present application. It should be understood that the above description is only the specific embodiments of the present application and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present application should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining at least two pieces of image information including color information and depth information of a target scene, where the at least two pieces of image information are respectively collected from different angles by at least two pre-deployed image acquisition devices; Respectively obtaining pictures of at least three objects from each of the at least two pieces of image information, and each of the at least three objects is included in the at least two pieces of image information; For each object among the at least three objects, setting a calibration point in each of the at least two pieces of image information to obtain the spatial coordinates of each calibration point corresponding to each piece of image information; For each object among the at least three objects, based on the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information, obtaining the relationship features between the at least two pieces of image information, and the relationship features between the two pieces of image information include: the offset matrix between the two pieces of image information; Based on the relationship features between each piece of image information and a reference image, converting the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image to obtain a target image, and the reference image is any one of the at least two pieces of image information; When the number of image acquisition devices is two, the two image acquisition devices are respectively denoted as K1 device and K2 device. In the image information corresponding to the K1 device and the K2 device respectively, select three objects A, B, and C that appear in the scene, and A, B, and C are simultaneously captured in the respective corresponding image information by the K1 device and the K2 device in the same image frame. Taking A, B, and C as calibration points, then in the image information obtained by the K1 device and the K2 device respectively, the three points A, B, and C form two triangles respectively. Denote the triangle formed in the image information corresponding to the K1 device as ΔA0B0C0, and denote the triangle formed in the image information corresponding to the K2 device as ΔA1B1C1. According to the coordinate differences of each vertex in the two pieces of image information, obtain the offset matrix M between the two pieces of image information, and the offset matrix M is the relationship feature. The calculation process of the offset matrix M specifically includes: According to the vector and The rotation from the former to the latter is represented by the quaternion q1; Let a perpendicular line to AB be drawn through point C, with the foot of the perpendicular being P. Vectors are calculated respectively according to the coordinates of each triangle The vector obtained after the q1 transformation Calculated from to The rotation is denoted as the quaternion q2; Multiplying q1 by q2 to obtain the final rotation coefficient r, and after rotating by r, the two triangles are completely parallel; The left multiplication of \(r\) on \(A0\) guarantees \(A'0\), and the displacement vector is denoted as \(t\). Based on the displacement vector t, the rotation coefficient r, and the scaling coefficient s, obtaining the offset matrix M between the image information corresponding to the K1 device and the image information corresponding to the K2 device; Repeating the operation process for the BC side and the AC side, and performing mean processing after obtaining the respective corresponding offset matrices; When the number of image processing devices is greater than 2, repeating the above process for each device to obtain the offset matrix M between every two image devices; 2. The image processing method according to claim 1, wherein For the pictures of the at least three objects, the pictures of all the objects come from the image information of the same image frame; 3. The image processing method according to claim 1, wherein Before obtaining the pictures of the at least three objects, the image processing method further includes establishing the corresponding relationship between the color information and the depth information according to a pre-set correction matrix.

4. The image processing method according to claim 1, wherein The described image processing method further includes: after obtaining a target image, judging the target image according to a preset rule and obtaining a corresponding result value. If the result value corresponding to the target image is within a preset threshold range, it is output as the final result; If the result value corresponding to the target image is not within the preset threshold range, pictures of at least three new objects are re-obtained from at least two pieces of image information. Each of the at least three new objects is included in the at least two pieces of image information and is different from at least three objects corresponding to the image frame of the target image.

5. An image processing apparatus, characterized in that, The described image processing device is used to implement the image processing method according to any one of claims 1-4. The device includes: A first acquisition module, configured to acquire at least two pieces of image information of a target scene, where the at least two pieces of image information are respectively acquired from different angles by at least two pre-deployed image acquisition devices; the image information includes color information and depth information; A second acquisition module, configured to respectively acquire pictures of at least three objects from each of the at least two pieces of image information, and each of the at least three objects is included in the at least two pieces of image information; A calibration module, configured to set a calibration point in each of the at least two pieces of image information for each of the at least three objects, and obtain the spatial coordinates of each calibration point corresponding to each piece of image information; A first processing module, configured to obtain the relationship features between at least two pieces of image information for each of the at least three objects according to the spatial coordinates of the calibration points corresponding to each piece of image information and the at least two pieces of image information; A second processing module, configured to convert the spatial coordinates of each object in the image information to the spatial coordinates of the corresponding object in the reference image according to the relationship features between each piece of image information and the reference image, to obtain a target image, and the reference image is any one of the at least two pieces of image information; Before acquiring the pictures of at least three objects, it further includes establishing a correspondence between the color information and the depth information according to a preset correction matrix, and performing denoising screening on abnormal depth information content in the image information.

6. An electronic device, characterized in that, The described electronic device includes: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the method according to any one of claims 1-4 by executing the computer instructions.

7. A computer-readable storage medium, characterized in that, The described computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images