Data reconstruction method and system

Through the mirror transformation technology of vehicle data reconstruction and acquisition, the data of the target vehicle is reconstructed using symmetric axisymmetric, the problem of high cost and limitations of vehicle data reconstruction in dynamic scenarios is solved, and the effective reconstruction of dynamic objects is achieved.

CN118967742APending Publication Date: 2024-11-15SUZHOU QINGZHOU ZHIHANG INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310545077.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, vehicle data reconstruction and acquisition is expensive, and it is difficult to achieve effective data acquisition in dynamic scenarios, resulting in great limitations in reconstruction and the inability to effectively deal with scene reconstruction of dynamic objects.

Method used

By collecting the first side of the target vehicle during the acquisition cycle, the acquisition position sequence and image sequence are acquired, and mirror transformation is performed to construct the second side and mirror position sequence, and the data of the target vehicle is reconstructed using symmetric axial symmetry. , achieve overall data reconstruction of the target vehicle.

Benefits of technology

It reduces the acquisition cost of vehicle data reconstruction, realizes effective data reconstruction in dynamic scenarios, and solves the limitations of dynamic object reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967742A_ABST
    Figure CN118967742A_ABST
Patent Text Reader

Abstract

The invention discloses a data reconstruction method and system. The method comprises the steps that in an acquisition period, a first side face of a target vehicle is acquired, an acquisition pose sequence is obtained, and the acquisition pose sequence comprises at least one pose for acquiring the target vehicle of acquisition equipment; determining a first image sequence obtained by performing image recognition on the first side surface by the acquisition device, the acquisition pose sequence corresponding to the first image sequence; mirror image transformation is conducted on the first image sequence, a second image sequence constructed for a second side face of the target vehicle is obtained, and the second side face and the first side face are symmetric based on a preset symmetric axis; performing mirror image transformation on the acquired pose sequence to obtain a constructed mirror image pose sequence for acquiring the second side surface; and obtaining a data reconstruction result of the target vehicle based on the mirror image pose sequence and the second image sequence. According to the invention, the technical problem that the acquisition cost of vehicle data reconstruction is not ideal in the prior art is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a data reconstruction method and system. Background Art

[0002] In addition to static natural scenes, the autonomous driving simulation system also has many dynamic vehicles. The current method of using neural radiation fields for three-dimensional reconstruction can better cope with the reconstruction problem of static scenes, but the reconstruction of scenes containing dynamic objects under the premise of a longer time and larger scenes has not yet been solved. In related technologies, vehicle data reconstruction is achieved by taking multi-perspective shots of the target vehicle to obtain images and postures from multiple perspectives for reconstruction. However, the acquisition cost of this process is relatively high and needs to be implemented in static scenes. The reconstruction has great limitations, and the car models are updated and iterated quickly, which requires repeated acquisition and updating, resulting in unsatisfactory data acquisition costs.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a data reconstruction method and system to at least solve the technical problem of unsatisfactory acquisition cost of vehicle data reconstruction existing in the related art.

[0005] According to one aspect of an embodiment of the present invention, a data reconstruction method is provided, comprising: in an acquisition cycle, acquiring a first side of a target vehicle to obtain an acquisition pose sequence, wherein the acquisition pose sequence includes at least one pose of the target vehicle acquired by an acquisition device; determining a first image sequence obtained by the acquisition device performing image recognition on the first side, wherein the acquisition pose sequence corresponds to the first image sequence; mirroring the first image sequence to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; mirroring the acquisition pose sequence to obtain a constructed mirror pose sequence for acquiring the second side; and obtaining a data reconstruction result of the target vehicle based on the mirror pose sequence and the second image sequence.

[0006] According to another aspect of an embodiment of the present invention, a data reconstruction system is provided, comprising: an acquisition device, which acquires a first side of a target vehicle during an acquisition cycle to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture of the target vehicle acquired by the acquisition device; an identification device, which determines a first image sequence obtained by the acquisition device performing image recognition on the first side, wherein the acquisition posture sequence corresponds to the first image sequence; a first mirror device, which performs a mirror transformation on the first image sequence to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; a second mirror device, which performs a mirror transformation on the acquisition posture sequence to obtain a constructed mirror posture sequence for acquiring the second side; and a reconstruction device, which obtains a data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence.

[0007] In an embodiment of the present invention, a mirror construction method is adopted, and a first side of the target vehicle is collected during a collection cycle to obtain a collection pose sequence, wherein the collection pose sequence includes at least one pose of the target vehicle collected by the collection device; a first image sequence is determined by the collection device performing image recognition on the first side, wherein the collection pose sequence corresponds to the first image sequence; the first image sequence is mirrored to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; the collection pose sequence is mirrored to obtain a constructed mirror pose sequence for collecting the second side; based on the mirror pose sequence and the second image sequence, the data reconstruction result of the target vehicle is obtained. The purpose of reconstructing the entire vehicle using the single-side data of the target vehicle is achieved, and the technical effect of reducing the collection cost during vehicle data reconstruction is achieved, thereby solving the technical problem of unsatisfactory collection cost of vehicle data reconstruction existing in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0009] Figure 1a is a schematic diagram of an alternative data reconstruction method;

[0010] Figure 1b is a schematic diagram of another optional data reconstruction method;

[0011] Figure 2 is a flowchart of an optional data reconstruction method provided according to an embodiment of the present invention;

[0012] Figure 3 is a schematic diagram of identification points of an optional data reconstruction method provided in an embodiment of the present invention;

[0013] Figure 4 is a mirror image schematic diagram of an optional data reconstruction method provided according to an embodiment of the present invention;

[0014] Figure 5 is a schematic flow chart of an optional data reconstruction method provided according to an embodiment of the present invention;

[0015] Figure 6 is a schematic diagram of an optional data reconstruction device provided according to an embodiment of the present invention;

[0016] Figure 7 It is a structural block diagram of an optional data reconstruction system provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] The 3D reconstruction technology in the field of autonomous driving can be used for automatic labeling and simulation applications. Among them, the automatic labeling technology can save a lot of labor for manual labeling, thereby automating the labeling of massive data and providing more labeled data for downstream tasks. The main reason why the current autonomous driving technology cannot operate in a completely unmanned state is that there are a large number of difficult boundary cases that are difficult to predict, and simulation technology can test these difficult scenarios at a low cost. In the related technology, since the neural radiation field needs to be processed with multi-angle acquisition data input, under static conditions, acquisition equipment at multiple angles can be arranged for acquisition, but in dynamic conditions, that is, there is relative motion between the target vehicle and the acquisition equipment, it is difficult to obtain image data on the side facing away from the acquisition equipment.

[0020] Therefore, current vehicle reconstruction relies on multi-view shooting of the target vehicle to obtain images and postures from multiple perspectives for reconstruction.

[0021] As for the reconstruction of dynamic objects in the autonomous driving scene, it involves two main objects: the ego vehicle (i.e., a movable device equipped with a collection device) and the target object. The present invention mainly discusses the reconstruction of the dynamic vehicle in the scene (hereinafter referred to as the target vehicle). This problem has the following two difficulties: First, the positions of the ego vehicle and the target vehicle in the world coordinate system change over time. Figure 1a is a schematic diagram of an optional data reconstruction method, such as Figure 1a As shown in the figure, cam represents the camera on the ego vehicle (i.e., acquisition device), and obj represents the target vehicle. The meeting between the ego vehicle and the target vehicle is a dynamic process, which is a position diagram in the world coordinate system. At the first moment t1, cam1 represents the position of the ego vehicle camera at the moment t1, and the corresponding target vehicle is at the position of obj1 at this time, and so on. Figure 1a The three moments are illustrated in Figure 2. The reconstructed point clouds from different viewpoints are located in different spatial positions and cannot form a complete vehicle.

[0022] Figure 1b is a schematic diagram of another optional data reconstruction method, such as Figure 1b As shown, it is a schematic diagram in the target vehicle coordinate system. If the data collected from the camera positions of cam1 to cam3 are input as a set of data, since the ego vehicle is always on one side of the target vehicle during the oncoming process, the data cannot be collected on the other side.

[0023] In response to the above problems, an embodiment of the present invention provides a method embodiment of data reconstruction. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 2 is a flow chart of a data reconstruction method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0025] Step S202, in a collection cycle, collecting a first side of the target vehicle to obtain a collection pose sequence, wherein the collection pose sequence includes at least one pose collected by the collection device for the target vehicle;

[0026] It can be understood that when collecting data from a target vehicle, since it is a dynamic process, only one side of the target vehicle can be collected, and there must be a visual blind spot, that is, the back of the target vehicle cannot be collected. At least one collection can be set in one collection cycle, corresponding to at least one collection posture, forming a collection posture sequence for the collection device to collect data from the target vehicle. By obtaining the visual angle information for collection, it is beneficial to use the neural radiation field for processing.

[0027] As an optional embodiment, the execution subject of the data reconstruction method may be a main controller, and there is a wired or wireless connection relationship between the main controller and the acquisition device.

[0028] It should be noted that the pose includes two types of information: position and attitude. The position can be represented by coordinates in space, and the attitude is the angle between a rigid body and each coordinate axis in the coordinate system, which is used to indicate the orientation of the rigid body.

[0029] Optionally, a variety of means may be used to collect data on the target vehicle, for example, a collection device installed in a movable device, where the collection device may be a camera or a video camera.

[0030] In an optional embodiment, during the acquisition cycle, the first side of the target vehicle is acquired to obtain an acquisition pose sequence, including: a first world pose sequence based on the target vehicle in the world coordinate system, and a second world pose sequence of the acquisition device in the world coordinate system; during the acquisition cycle, based on the first world pose sequence and the second world pose sequence, the acquisition pose sequence in the target vehicle coordinate system is determined.

[0031] It can be understood that the world coordinate system represents the absolute position relationship of the area where the target vehicle is located. Compared with the world coordinate system, the target vehicle coordinate system is a relative coordinate system. Target vehicles at different positions correspond to different target vehicle coordinate systems. This is because the target vehicle coordinate system is established with the position coordinates of the target vehicle as the origin, and the target vehicle coordinate system is oriented with the posture of the target vehicle. Based on the first world pose sequence and the second world pose sequence in the world coordinate system, the acquisition pose sequence of the target vehicle acquired by the acquisition device is determined, that is, the acquisition pose sequence is obtained by coordinate conversion based on the first world pose sequence and the second world pose sequence in the world coordinate system. Through the above processing, the target vehicle coordinate system established with the target vehicle and the acquisition pose sequence of the acquisition device in the target vehicle coordinate system are obtained, which is conducive to the subsequent construction of the mirror image perspective in the target vehicle coordinate system to obtain the mirror pose sequence.

[0032] Step S204, determining a first image sequence obtained by the acquisition device performing image recognition on the first side surface, wherein the acquisition posture sequence corresponds to the first image sequence;

[0033] It can be understood that the first image sequence is obtained through actual measurement, and is different from the constructed mirror image posture sequence, and is obtained by performing image recognition using a capture device that captures the corresponding posture sequence.

[0034] In an optional embodiment, the above-mentioned determination of the first image sequence obtained by the above-mentioned acquisition device performing image recognition on the above-mentioned first side includes: in the above-mentioned acquisition cycle, obtaining the above-mentioned acquisition device performing image acquisition on the above-mentioned first side using the above-mentioned acquisition posture sequence to obtain an initial image sequence, wherein the image range of the above-mentioned initial image sequence is larger than the image range of the above-mentioned first image sequence; and using a predetermined image recognition algorithm to perform image recognition on the above-mentioned initial image sequence to obtain the above-mentioned first image sequence.

[0035] It can be understood that when capturing the target vehicle, the background where the target vehicle is located and other objects included in the background will be captured. At this time, the initial image sequence is obtained, and it is necessary to separate it from the target vehicle to obtain the first image sequence. In other words, the image range of the initial image sequence will be larger than the image range of the first image sequence. The initial image sequence is recognized using a predetermined image recognition algorithm to obtain the first image sequence for mirror processing.

[0036] In an optional embodiment, the above-mentioned predetermined image recognition algorithm is used to perform image recognition on the above-mentioned initial image sequence to obtain the above-mentioned first image sequence, including: when the above-mentioned image recognition algorithm includes an instance segmentation method, the instance with the semantics of a vehicle in the above-mentioned initial image sequence is recognized to determine the initial recognition result of the above-mentioned target vehicle; based on the above-mentioned initial recognition result, the above-mentioned first image sequence is obtained.

[0037] It can be understood that when the initial image sequence is processed, the target vehicle is separated from irrelevant objects and other vehicles in the background, and the target vehicle is determined in the instance with the semantics of a vehicle as the initial recognition result of the target vehicle. Based on the above initial recognition result, the first image sequence is obtained.

[0038] In an optional embodiment, the above-mentioned identifying instances with semantics of vehicles in the above-mentioned initial image sequence to determine the initial recognition result of the above-mentioned target vehicle includes: when multiple instances with semantics of vehicles are identified and there is overlap between the multiple instances, determining a target bounding box and a center point of the above-mentioned target bounding box; and determining an instance including the above-mentioned center point among the above-mentioned multiple instances as the above-mentioned target vehicle.

[0039] It can be understood that in the initial image sequence acquired, there may be overlap between the target vehicle and other vehicles. In order to improve the accuracy of target vehicle recognition, the symmetry of the vehicle is used to identify the center point of the target bounding box of the target vehicle, which is the center point of the target vehicle. The instance in which the above center point falls can be determined as the target vehicle. Through the above processing, the common image overlap problem in actual application scenarios can be solved, and even if the target vehicle is blocked, the accuracy of target vehicle recognition can be improved.

[0040] In an optional embodiment, the above-mentioned first image sequence is obtained based on the above-mentioned initial recognition result, including: when the above-mentioned image recognition algorithm includes a three-dimensional tracking frame, the above-mentioned initial recognition result is processed to obtain a target three-dimensional frame for tracking the above-mentioned target vehicle; using the above-mentioned target three-dimensional frame, the above-mentioned first image sequence is determined in the above-mentioned initial image sequence.

[0041] It can be understood that in order to improve the recognition accuracy of the target vehicle and avoid including only irrelevant objects, a three-dimensional tracking frame is used for further processing, and a target three-dimensional frame for tracking the target vehicle is obtained based on the initial recognition result. The target three-dimensional frame is used to determine a first image sequence with relatively ideal accuracy in the initial image sequence. Through the above processing, the instance segmentation method can be combined with the three-dimensional tracking frame to improve the accuracy of the first image sequence and avoid including irrelevant objects, which leads to unsatisfactory reconstruction results.

[0042] In an optional embodiment, the initial image sequence is an image sequence in a target vehicle coordinate system, and the first image sequence is an image sequence in an acquisition coordinate system, wherein the target vehicle coordinate system is a coordinate system with the position of the target vehicle in the world coordinate system as the origin and the orientation of the target vehicle to the acquisition device as the posture, and the acquisition coordinate system is a coordinate system with the position of the acquisition device in the world coordinate system as the origin and the orientation of the acquisition device to the target vehicle as the posture.

[0043] It can be understood that the above-mentioned initial image sequence exists in the target vehicle coordinate system. The target vehicle coordinate system is a coordinate system constructed with the position of the target vehicle in the world coordinate system as the origin and the orientation of the target vehicle to the acquisition device as the posture. In other words, the acquisition posture sequence is also in the target vehicle coordinate system in the same way as the initial image sequence. The first image sequence is obtained by image recognition and coordinate transformation from the initial image sequence and is in the acquisition coordinate system. The above-mentioned acquisition coordinate system is a coordinate system constructed with the position of the acquisition device in the world coordinate system as the origin and the orientation of the acquisition device to the target vehicle. It should be noted that the origin setting of the target vehicle coordinate system is different from that of the acquisition coordinate system, and the orientation is also different. The orientation of the target vehicle to the acquisition device and the orientation of the acquisition device to the target vehicle are two opposite orientations.

[0044] It still needs to be explained that the significance of obtaining the first image sequence by transforming the initial image sequence through the coordinate system is that the initial image sequence is obtained based on the acquisition posture sequence. Since the neural radiation field requires the acquisition of viewing angle and image data to be reconstructed, the focus is on obtaining the acquisition posture sequence that represents the viewing angle, which is used to subsequently construct a mirror posture sequence for capturing the second side.

[0045] In an optional embodiment, the above-mentioned initial recognition result is processed to obtain a target three-dimensional frame for tracking the above-mentioned target vehicle, including: converting the above-mentioned acquisition posture sequence to obtain a target vehicle posture sequence corresponding to the above-mentioned target vehicle in the above-mentioned acquisition coordinate system; using the above-mentioned three-dimensional tracking frame method to process the above-mentioned initial recognition result, and determine the initial three-dimensional frame for tracking the above-mentioned target vehicle in the above-mentioned target vehicle coordinate system; based on the above-mentioned initial three-dimensional frame and the above-mentioned target vehicle posture sequence, determine the above-mentioned target three-dimensional frame in the above-mentioned acquisition coordinate system.

[0046] It can be understood that the acquisition posture sequence is in the target vehicle coordinate system. After the coordinate system conversion, the target vehicle posture sequence corresponding to the target vehicle in the acquisition coordinate system can be obtained. In order to reduce the impact of vehicles very close to the target vehicle on the reconstruction result of the target vehicle, further selection is performed with the help of a three-dimensional tracking frame method. The initial recognition result is processed, and the initial three-dimensional frame for tracking the target vehicle is determined in the target vehicle coordinate system. The same coordinate system conversion method is used to transform the initial three-dimensional frame. Based on the initial three-dimensional frame and the target vehicle posture sequence, the target three-dimensional frame for tracking the target vehicle in the acquisition coordinate system is determined. Through the above processing, the initial three-dimensional frame is conducive to improving the pixel capture accuracy of the target vehicle, in order to convert it to the acquisition coordinate system for processing, and then obtain the target three-dimensional frame. Compared with only performing the instance segmentation method to determine the target vehicle and then intercepting the image data, the use of a three-dimensional frame tracking method is conducive to obtaining better accuracy.

[0047] In an optional embodiment, the above-mentioned target three-dimensional frame in the acquisition coordinate system is determined based on the above-mentioned initial three-dimensional frame and the above-mentioned target vehicle posture sequence, including: determining a candidate three-dimensional frame in the above-mentioned acquisition coordinate system based on the above-mentioned initial three-dimensional frame and the above-mentioned target vehicle posture sequence; when the range of the above-mentioned candidate three-dimensional frame is larger than the image range of the above-mentioned target vehicle, reducing the above-mentioned candidate three-dimensional frame to obtain the above-mentioned target three-dimensional frame, so that the range of the above-mentioned target three-dimensional frame is less than or equal to the image range of the above-mentioned target vehicle.

[0048] It can be understood that based on the initial 3D frame and the target vehicle position sequence, the candidate 3D frame in the acquisition coordinate system is determined. If the candidate 3D frame is included in the first 3D frame, it will cause reconstruction errors. Therefore, when the range of the candidate 3D frame is larger than the image range of the target vehicle, it is necessary to reduce the candidate 3D frame to obtain the target 3D frame so that the target 3D frame falls within the image range of the target vehicle. Through the above processing, the possibility of including irrelevant objects can be minimized, and the accuracy and efficiency of vehicle data reconstruction can be improved.

[0049] In an optional embodiment, the above-mentioned target three-dimensional frame in the above-mentioned acquisition coordinate system is determined based on the above-mentioned initial three-dimensional frame and the above-mentioned target vehicle pose sequence, including: inverting the above-mentioned acquisition pose sequence to obtain the target vehicle pose sequence of the above-mentioned target vehicle in the above-mentioned acquisition coordinate system; based on the above-mentioned target vehicle pose sequence and the above-mentioned initial three-dimensional frame, the above-mentioned target three-dimensional frame is obtained.

[0050] It can be understood that in order to convert the target vehicle coordinate system and the acquisition coordinate system, the target vehicle position sequence in the target vehicle coordinate system is inverted to obtain the inverted target vehicle position sequence corresponding to the target vehicle in the acquisition coordinate system. The initial three-dimensional frame is also in the target vehicle coordinate system and needs to be converted to obtain the target three-dimensional frame. The coordinate conversion is completed through the above processing to obtain the target three-dimensional frame.

[0051] For ease of understanding, a specific example is given. For example, a pose in the target vehicle pose sequence in the target vehicle coordinate system is recorded as right To invert, in, The coordinates of the origin of the target vehicle coordinate system (which can be understood as the position of the target vehicle in the world coordinate system) P o =(0, 0, 0), the origin of the acquisition coordinate system (which can be understood as the position of the acquisition device in the world coordinate system) is denoted as P c ,use P o With P c The conversion relationship between them is Similarly, other points in the target vehicle coordinate system can be converted to the acquisition coordinate system in the same way. A cuboid is generated based on the length, width, and height of the initial 3D box, and each point in the cuboid is transformed in the same way as the coordinate origin (e.g., each point is multiplied by ) to obtain the target three-dimensional frame in the acquisition coordinate system.

[0052] In an optional embodiment, the above-mentioned target three-dimensional frame is used to determine the above-mentioned first image sequence in the above-mentioned initial image sequence, including: obtaining the focal length and optical center of the above-mentioned acquisition device for acquiring the above-mentioned target vehicle; based on the above-mentioned focal length and the above-mentioned optical center, obtaining the projected target three-dimensional frame; based on the above-mentioned projected target three-dimensional frame, selecting the above-mentioned first image sequence in the above-mentioned initial image sequence.

[0053] It can be understood that the focal length and the optical center are acquisition parameters for acquiring the target vehicle. According to the focal length and the optical center, the target three-dimensional frame is projected, and converted from three dimensions to two dimensions to obtain the target two-dimensional frame. Through the above processing, considering the focal length and the optical center for projection, it is beneficial to improve the accuracy of the first image sequence selected in the initial image sequence.

[0054] It should be noted that the image data captured by the projected target 3D frame can verify the accuracy of the 3D pose transformation and the accuracy of the sensor data for image acquisition. If any error occurs in the sensor and the 3D pose transformation, the image data captured by the projected target 3D frame will not need to be reconstructed when there is an error in the first image sequence, thereby improving the processing efficiency of reconstructing the target vehicle.

[0055] In an optional embodiment, after identifying the instances with semantics of vehicles in the above-mentioned initial image sequence and determining the initial recognition result of the above-mentioned target vehicle, the above-mentioned method further includes: when the above-mentioned initial recognition result is a partial image of the target vehicle, based on the above-mentioned partial image of the target vehicle, determining at least one identification point of the above-mentioned target vehicle, wherein the above-mentioned at least one identification point is at least one of a preset identification point set; based on the above-mentioned at least one identification point, updating the above-mentioned initial image sequence so that the updated initial recognition result includes the above-mentioned identification point set.

[0056] It can be understood that after obtaining the initial recognition result, due to the limited image size, there may be a situation where the acquisition of the target vehicle is incomplete, that is, the initial recognition result is a partial image of the target vehicle. In order to avoid the effect of vehicle data reconstruction due to incomplete data acquisition, it is necessary to determine whether and how to reacquire the initial image sequence. Based on the partial image of the target vehicle, at least one identification point of the target vehicle can be determined, and a preset identification point set is set for the target vehicle to determine the part of the target vehicle. Based on the at least one identification point determined above, the image of the first side is re-captured, and the initial image sequence can be updated so that the updated initial recognition result is complete, that is, the identification point set is obtained in the updated initial recognition result.

[0057] It should be noted that the at least one identification point may be of multiple types, for example: the front point, the center point, and the rear point of the target vehicle. Figure 3 is a schematic diagram of identification points of an optional data reconstruction method according to an embodiment of the present invention, such as Figure 3 As shown, the white outline box is a schematic diagram of the target vehicle. The target vehicle is identified, and the front point is recorded as a circle, the center point is recorded as a triangle, and the rear point is recorded as a square. The vehicle body contour can be identified through the above three identification points to determine whether the collected image block is complete.

[0058] In an optional embodiment, based on the at least one identification point, the initial image sequence is updated, including: when the front point of the target vehicle is identified, determining to adjust the first angle toward the rear of the vehicle to obtain a new initial image; when the front point and center point of the target vehicle are identified, determining to adjust the second angle toward the rear of the vehicle to obtain a new initial image, wherein the first angle is greater than the second angle; when the rear point of the target vehicle is identified, determining to adjust the third angle toward the front of the vehicle to obtain a new initial image; when the rear point and center point of the target vehicle are identified, determining to adjust the fourth angle toward the front of the vehicle to obtain a new initial image, wherein the third angle is greater than the fourth angle.

[0059] It can be understood that according to different identified marking points, the acquisition angle when re-capturing images is adjusted accordingly to improve the efficiency of acquiring the initial image sequence and avoid the situation where the re-acquired initial image sequence still does not include the complete target vehicle.

[0060] Step S206, performing a mirror transformation on the first image sequence to obtain a second image sequence constructed for a second side surface of the target vehicle, wherein the second side surface is symmetrical to the first side surface based on a predetermined symmetry axis;

[0061] It can be understood that the mirroring processing of the first image data utilizes the symmetry of the vehicle along the long axis direction. The long axis of the vehicle, i.e., the line connecting the front of the vehicle and the parking space, can be used as the predetermined symmetry axis. Since the first side surface and the second side surface are symmetrical along the predetermined symmetry axis, the image data of the second side surface can be obtained by mirroring to generate a constructed second image sequence.

[0062] Optionally, the above-mentioned image data may be of multiple types, for example, the image includes RGB (red, green and blue primary colors) data of pixels.

[0063] In an optional embodiment, the first image sequence is mirrored to obtain a second image sequence constructed for the second side of the target vehicle, including: based on the center point of the target vehicle, centrally symmetric the optical center for capturing the target vehicle to obtain a mirror optical center; determining the central axis in the direction of the line connecting the front and rear of the target vehicle; based on the central axis and the mirror optical center, flipping the first image sequence to obtain the second image sequence.

[0064] It can be understood that in order to perform reasonable reconstruction, the optical center of the acquisition parameters needs to change with the acquisition posture, and the optical center of the mirror image is obtained through central symmetry with the center point of the vehicle. In order to utilize the characteristics of vehicle symmetry along the long axis, the central axis in the direction of the line between the front and rear of the target vehicle is determined. Based on the central axis and the optical center of the mirror image, the first image sequence is flipped to obtain the second image sequence.

[0065] Step S208, performing a mirror transformation on the above-mentioned collected posture sequence to obtain a constructed mirror posture sequence for collecting the second side surface;

[0066] It can be understood that the mirror processing is performed in the target vehicle coordinate system with the position coordinate of the target vehicle as the origin, and thus a mirror pose sequence is obtained. It should be noted that the above-mentioned mirror pose sequence is not obtained by actual measurement, but is constructed.

[0067] In an optional embodiment, the above-mentioned mirroring transformation of the acquired posture sequence to obtain a mirror posture sequence includes: determining the acquired position sequence and the acquired Euler angle sequence included in the acquired posture sequence; mirroring the acquired Euler angle sequence to obtain a mirror Euler angle sequence; mirroring the acquired position sequence to obtain a mirror position sequence; and determining the mirror posture sequence based on the mirror Euler angle sequence and the mirror position sequence.

[0068] It can be understood that each collected posture in the collected posture sequence corresponds to a collected position and a collected Euler angle, and a collected position sequence and a collected Euler angle sequence are obtained. Euler angles are used to mark the rotation of an object. Since the posture includes position information and orientation information, it is composed of a rotation matrix and position coordinates. By obtaining a mirror Euler angle sequence, the rotation matrix included in the collected posture sequence is processed to obtain a rotation matrix after mirroring. In order to obtain a mirror posture sequence, it is also necessary to obtain the position coordinates after mirroring, that is, a mirror position sequence. Based on the mirror Euler angle and the mirror position sequence, a mirror posture sequence is obtained together. Through the above processing, a mirror posture sequence for representing the mirror viewing angle is constructed in the target vehicle coordinate system, which is conducive to reconstructing the second side.

[0069] For ease of understanding, a specific example is given, for example: the position coordinate corresponding to a pose in the captured pose sequence of the target vehicle is recorded as (T x , T y , T z ) and the Euler angle is denoted as (α x , α y , α z ), correspondingly, the above pose in the target vehicle coordinate system is denoted as T:

[0070]

[0071] in, is the rotation matrix included in the above posture, and x, y, and z represent the x-axis, y-axis, and z-axis respectively.

[0072] According to the characteristics of mirror transformation, the position coordinates of the above posture after the mirror image are obtained as (T x , -Ty , T z ) and the mirror image Euler angle is recorded as (-α x , α y , -α z ). Correspondingly, the above-mentioned pose after mirroring constructed in the target vehicle coordinate system is recorded as

[0073]

[0074] in, is the rotation matrix included in the above-mentioned pose after mirroring. Through the above processing, the above-mentioned pose and the above-mentioned pose after mirroring are obtained, which are arranged symmetrically along the first side and the second side of the target side with the position coordinates of the target vehicle as the origin. In a similar way, the poses included in the acquisition pose sequence are processed in the same way to obtain the mirrored pose corresponding to each pose, and generate the acquisition pose sequence and the mirrored pose sequence symmetrically arranged along the first side and the second side of the target side.

[0075] Figure 4 is a mirror image schematic diagram of an optional data reconstruction method according to an embodiment of the present invention, such as Figure 4 As shown, a collection pose sequence and a mirror pose sequence are shown which are symmetrically arranged along the first side and the second side of the target side. Cam1...3 represent the collection pose sequence of the collection device for collecting the target vehicle (ie, obj) at the corresponding time, and cam′1...3 represent the constructed mirror pose sequence. Figure 4 The x-axis is parallel to the long axis of the target vehicle, and the z-axis is perpendicular to the paper and upward.

[0076] Step S210, obtaining a data reconstruction result of the target vehicle based on the mirror image pose sequence and the second image sequence;

[0077] It can be understood that the mirror pose sequence is constructed by acquiring the pose sequence, and is used to characterize the acquisition viewing angle of the second side. The second image sequence is constructed by the first image sequence, and is used to characterize the image data of the second side. Based on the mirror pose sequence and the second image sequence, the data reconstruction result of the target vehicle is obtained.

[0078] Optionally, a neural radiation field is used to obtain data reconstruction results.

[0079] Through the above steps S202 to S210, the purpose of reconstructing the entire vehicle by utilizing the single-side data of the target vehicle can be achieved, achieving the technical effect of reducing the acquisition cost during vehicle data reconstruction, thereby solving the technical problem of unsatisfactory acquisition cost of vehicle data reconstruction existing in related technologies.

[0080] The acquisition cost of vehicle data reconstruction in the related art is not ideal. Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation method, which uses a camera (i.e., acquisition device) installed on the vehicle to acquire data of the target vehicle and complete the reconstruction of the target vehicle. Figure 5 FIG. 1 is a schematic flow chart of an optional data reconstruction method according to an embodiment of the present invention, which is specifically described by the following steps:

[0081] Step s1: record the camera pose in the world coordinate system (i.e., one of the second world pose sequences) as T c , the target car pose (i.e. one of the first world pose sequences) is denoted as T obj , then the camera pose in the target vehicle coordinate system (i.e. one of the acquired pose sequences) is recorded as

[0082]

[0083] By T c and T obj Mathematical transformations can yield

[0084] Step S2: according to the camera position in the target vehicle coordinate system The length, width and height of the initial three-dimensional tracking frame for tracking the target vehicle in the target vehicle coordinate system are calculated and converted to the target three-dimensional frame in the camera coordinate system (ie, the acquisition coordinate system).

[0085] right The inversion is performed to obtain the target vehicle pose in the acquisition coordinate system (i.e., one of the target vehicle pose sequences), and the points in the target vehicle coordinate system are transformed in the same way to obtain each point corresponding to the acquisition coordinate system.

[0086] A pose in the target vehicle pose sequence in the target vehicle coordinate system is denoted as right To invert, in, The coordinates of the origin of the target vehicle coordinate system (which can be understood as the position of the target vehicle in the world coordinate system) P o =(0, 0, 0), the origin of the acquisition coordinate system (which can be understood as the position of the acquisition device in the world coordinate system) is denoted as P c ,use P o With P c The conversion relationship between them is Similarly, other points in the target vehicle coordinate system can be converted to the acquisition coordinate system in the same way. A cuboid is generated based on the length, width, and height of the initial 3D box, and each point in the cuboid is transformed in the same way as the coordinate origin (e.g., each point is multiplied by ) to obtain the target three-dimensional frame in the acquisition coordinate system.

[0087] Step S3, combining camera internal parameters such as focal length and optical center, projecting the above target 3D frame into the 2D image. When the projected target 3D frame is used for frame selection, image data including other vehicles or irrelevant objects is obtained, and the target 3D frame needs to be re-acquired.

[0088] Step S4, performing instance segmentation on the initial image sequence captured by the camera, intercepting all instances with vehicle semantics as initial recognition results, and intercepting corresponding image data in combination with the projected target three-dimensional frame to generate a first image sequence corresponding to the acquired posture sequence, so that the first image sequence only includes complete image data of the target vehicle.

[0089] Step S5, mirroring the image data of the target vehicle determined in steps S3 and S4, and processing the optical center position during the mirroring, flipping the optical center position symmetrically along the center point of the target vehicle to obtain a second image sequence after the mirror transformation.

[0090] Step S6, the camera position obtained in step S1 Perform mirror transformation to construct a mirror pose sequence corresponding to the symmetric perspective.

[0091] Step S7, sending the results of the above steps S5 and S6 into the neural radiation field for reconstruction, and after obtaining the mirror image posture sequence of the target vehicle and the corresponding second image sequence, obtaining the vehicle reconstruction result of the target vehicle.

[0092] The above optional implementation achieves at least the following effects: analyzing the geometric and texture features of the vehicle, which has the characteristic of being symmetrical about the long axis, and proposing to perform a series of posture transformations and symmetrical perspective construction on the images captured during the driving process of the vehicle, that is, the images containing the target vehicle taken when meeting the target vehicle and the corresponding camera posture of the vehicle when capturing the target vehicle, to enhance the original data (that is, the single-side data of the target vehicle), thereby achieving the purpose of obtaining surround perspective images and camera postures.

[0093] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0094] In this embodiment, a data reconstruction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0095] According to an embodiment of the present invention, there is also provided an apparatus embodiment for implementing the data reconstruction method. Figure 6 is a schematic diagram of a data reconstruction device according to an embodiment of the present invention. Figure 6 As shown, the data reconstruction device includes: an acquisition module 602, an identification module 604, a first mirror module 606, a second mirror module 608, and a reconstruction module 610. The device is described below.

[0096] A collection module 602 is used to collect a first side of the target vehicle during a collection period to obtain a collection pose sequence, wherein the collection pose sequence includes at least one pose collected by a collection device on the target vehicle;

[0097] The recognition module 604 is connected to the acquisition module 602 and is used to determine a first image sequence obtained by the acquisition device performing image recognition on the first side surface, wherein the acquisition posture sequence corresponds to the first image sequence;

[0098] A first mirror module 606, connected to the recognition module 604, is used to perform a mirror transformation on the first image sequence to obtain a second image sequence constructed for the second side surface of the target vehicle, wherein the second side surface is symmetrical with the first side surface based on a predetermined symmetry axis;

[0099] A second mirroring module 608, connected to the first mirroring module 606, is used to perform a mirroring transformation on the above-mentioned acquisition posture sequence to obtain a constructed mirrored posture sequence for acquiring the second side surface;

[0100] The reconstruction module 610 is connected to the second mirror module 608 and is used to obtain the data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence.

[0101] In a data reconstruction device provided by an embodiment of the present invention, an acquisition module 602 is used to acquire a first side of a target vehicle during an acquisition cycle to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture acquired by the acquisition device for the target vehicle; an identification module 604 is connected to the acquisition module 602, and is used to determine a first image sequence obtained by the acquisition device performing image recognition on the first side, wherein the acquisition posture sequence corresponds to the first image sequence; a first mirror module 606 is connected to the identification module 604, and is used to perform a mirror transformation on the first image sequence to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; a second mirror module 608 is connected to the first mirror module 606, and is used to perform a mirror transformation on the acquisition posture sequence to obtain a constructed mirror posture sequence for acquiring the second side; a reconstruction module 610 is connected to the second mirror module 608, and is used to obtain a data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence. The purpose of reconstructing the entire vehicle by using the single-side data of the target vehicle is achieved, and the technical effect of reducing the acquisition cost during vehicle data reconstruction is realized, thereby solving the technical problem of unsatisfactory acquisition cost of vehicle data reconstruction existing in related technologies.

[0102] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0103] It should be noted that the acquisition module 602, the identification module 604, the first mirror module 606, the second mirror module 608, and the reconstruction module 610 correspond to steps S102 to S110 in the embodiment, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the modules as part of the device can be run in a computer terminal.

[0104] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0105] The above-mentioned data reconstruction device may also include a processor and a memory. The acquisition module 602, the identification module 604, the first mirror module 606, the second mirror module 608, the reconstruction module 610, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0106] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0107] A data reconstruction system is also provided in an embodiment of the present invention. The data reconstruction system provided in the embodiment of the present invention is introduced below. Figure 7 is a structural block diagram of an optional data reconstruction system according to an embodiment of the present invention. Figure 7 As shown, the system includes: a collection device 702, a recognition device 704, a first mirror device 706, a second mirror device 708, and a reconstruction device 710, and the system is described below.

[0108] The acquisition device 702 is used to acquire a first side surface of the target vehicle during an acquisition cycle to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture acquired by the acquisition device for the target vehicle;

[0109] The recognition device 704 is connected to the acquisition device 702 and is used to determine a first image sequence obtained by the acquisition device performing image recognition on the first side surface, wherein the acquisition posture sequence corresponds to the first image sequence;

[0110] A first mirror device 706, connected to the recognition device 704, is used to perform a mirror transformation on the first image sequence to obtain a second image sequence constructed for the second side surface of the target vehicle, wherein the second side surface is symmetrical with the first side surface based on a predetermined symmetry axis;

[0111] The second mirror device 708 is connected to the first mirror device 706 and is used to perform a mirror transformation on the above-mentioned acquisition posture sequence to obtain a constructed mirror posture sequence for acquiring the second side surface;

[0112] The reconstruction device 710 is connected to the second mirror device 708 and is used to obtain the data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence.

[0113] In a data reconstruction system provided by an embodiment of the present invention, an acquisition device 702 is used to acquire a first side of a target vehicle during an acquisition cycle to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture acquired by the acquisition device for the target vehicle; an identification device 704 is connected to the acquisition device 702, and is used to determine a first image sequence obtained by the acquisition device performing image recognition on the first side, wherein the acquisition posture sequence corresponds to the first image sequence; a first mirror device 706 is connected to the identification device 704, and is used to perform a mirror transformation on the first image sequence to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; a second mirror device 708 is connected to the first mirror device 706, and is used to perform a mirror transformation on the acquisition posture sequence to obtain a constructed mirror posture sequence for acquiring the second side; a reconstruction device 710 is connected to the second mirror device 708, and is used to obtain a data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence. The purpose of reconstructing the entire vehicle by using the single-side data of the target vehicle is achieved, thereby realizing the technical effect of reducing the acquisition cost during vehicle data reconstruction, and further solving the technical problem of unsatisfactory acquisition cost of vehicle data reconstruction existing in related technologies.

[0114] An embodiment of the present invention provides a non-volatile storage medium on which a program is stored. When the program is executed by a processor, a data reconstruction method is implemented.

[0115] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: in an acquisition cycle, the first side of the target vehicle is acquired to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture acquired by the acquisition device for the target vehicle; a first image sequence is determined by the acquisition device for image recognition of the first side, wherein the acquisition posture sequence corresponds to the first image sequence; the first image sequence is mirrored to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; the acquisition posture sequence is mirrored to obtain a constructed mirror posture sequence for acquiring the second side; based on the mirror posture sequence and the second image sequence, a data reconstruction result of the target vehicle is obtained. The device in this article may be a server, a PC, etc.

[0116] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: in an acquisition cycle, the first side of the target vehicle is acquired to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture of the target vehicle acquired by the acquisition device; a first image sequence is determined to be obtained by the acquisition device performing image recognition on the first side, wherein the acquisition posture sequence corresponds to the first image sequence; the first image sequence is mirrored to obtain a second image sequence constructed for the second side of the target vehicle, wherein the second side is symmetrical with the first side based on a predetermined symmetry axis; the acquisition posture sequence is mirrored to obtain a constructed mirror posture sequence for acquiring the second side; based on the mirror posture sequence and the second image sequence, a data reconstruction result of the target vehicle is obtained.

[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0119] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0125] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0126] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A data reconstruction method, characterized in that: include: In a collection cycle, a first side of the target vehicle is collected to obtain a collection pose sequence, wherein the collection pose sequence includes at least one pose collected by the collection device for the target vehicle; Determine a first image sequence obtained by the acquisition device performing image recognition on the first side, wherein the acquisition posture sequence corresponds to the first image sequence; Performing a mirror transformation on the first image sequence to obtain a second image sequence constructed for a second side surface of the target vehicle, wherein the second side surface is symmetrical to the first side surface based on a predetermined symmetry axis; Performing a mirror transformation on the acquisition posture sequence to obtain a constructed mirror posture sequence for acquiring the second side surface; Based on the mirror image pose sequence and the second image sequence, a data reconstruction result of the target vehicle is obtained.

2. The method according to claim 1, characterized in that The determining of a first image sequence obtained by performing image recognition on the first side by the acquisition device includes: In the acquisition cycle, the acquisition device acquires images of the first side surface by adopting the acquisition posture sequence to obtain an initial image sequence, wherein an image range of the initial image sequence is larger than an image range of the first image sequence; A predetermined image recognition algorithm is used to perform image recognition on the initial image sequence to obtain the first image sequence.

3. The method according to claim 2, characterized in that The step of using a predetermined image recognition algorithm to perform image recognition on the initial image sequence to obtain the first image sequence includes: In the case where the image recognition algorithm includes an instance segmentation method, identifying instances with semantics of vehicles in the initial image sequence to determine an initial recognition result of the target vehicle; Based on the initial recognition result, the first image sequence is obtained.

4. The method according to claim 3, characterized in that The obtaining the first image sequence based on the initial recognition result includes: In the case where the image recognition algorithm includes a three-dimensional tracking frame, processing the initial recognition result to obtain a target three-dimensional frame for tracking the target vehicle; The target three-dimensional frame is used to determine the first image sequence in the initial image sequence.

5. The method according to claim 4, characterized in that The initial image sequence is an image sequence in the target vehicle coordinate system, and the first image sequence is an image sequence in the acquisition coordinate system, wherein the target vehicle coordinate system is a coordinate system with the position of the target vehicle in the world coordinate system as the origin and the orientation of the target vehicle to the acquisition device as the posture, and the acquisition coordinate system is a coordinate system with the position of the acquisition device in the world coordinate system as the origin and the orientation of the acquisition device to the target vehicle as the posture.

6. The method according to claim 5, characterized in that The processing of the initial recognition result to obtain a target three-dimensional frame for tracking the target vehicle includes: Converting the acquired posture sequence to obtain a target vehicle posture sequence corresponding to the target vehicle in the acquired coordinate system; The initial recognition result is processed by adopting the three-dimensional tracking frame method, and an initial three-dimensional frame for tracking the target vehicle is determined in the target vehicle coordinate system; Based on the initial three-dimensional frame and the target vehicle position and posture sequence, the target three-dimensional frame in the acquisition coordinate system is determined.

7. The method according to claim 6, characterized in that The step of determining the target three-dimensional frame in the acquisition coordinate system based on the initial three-dimensional frame and the target vehicle position sequence includes: Determine a candidate three-dimensional frame in the acquisition coordinate system based on the initial three-dimensional frame and the target vehicle position sequence; When the range of the candidate three-dimensional frame is larger than the image range of the target vehicle, the candidate three-dimensional frame is reduced to obtain the target three-dimensional frame, so that the range of the target three-dimensional frame is smaller than or equal to the image range of the target vehicle.

8. The method according to claim 6, characterized in that The step of determining the target three-dimensional frame in the acquisition coordinate system based on the initial three-dimensional frame and the target vehicle position sequence includes: Inverting the acquired pose sequence to obtain a target vehicle pose sequence of the target vehicle in the acquired coordinate system; The target three-dimensional frame is obtained based on the target vehicle position sequence and the initial three-dimensional frame.

9. The method according to claim 6, characterized in that The step of using the target three-dimensional frame to determine the first image sequence in the initial image sequence includes: Obtaining the focal length and optical center of the target vehicle acquired by the acquisition device; Based on the focal length and the optical center, obtaining a projected target three-dimensional frame; Based on the projected target three-dimensional frame, the first image sequence is framed out from the initial image sequence.

10. A data reconstruction system, characterized in that: include: The acquisition device acquires a first side of the target vehicle during an acquisition cycle to obtain an acquisition posture sequence, wherein the acquisition posture sequence includes at least one posture acquired by the acquisition device on the target vehicle; A recognition device is configured to determine a first image sequence obtained by the acquisition device performing image recognition on the first side surface, wherein the acquisition posture sequence corresponds to the first image sequence; A first mirroring device is used to perform a mirroring transformation on the first image sequence to obtain a second image sequence constructed for a second side surface of the target vehicle, wherein the second side surface is symmetrical to the first side surface based on a predetermined symmetry axis; A second mirror device performs a mirror transformation on the captured pose sequence to obtain a constructed mirror pose sequence for capturing the second side surface; A reconstruction device obtains a data reconstruction result of the target vehicle based on the mirror posture sequence and the second image sequence.