Method and device for determining truncation proportion of object in image and electronic equipment
By generating BEV spatial images and calculating the truncation ratio in the vehicle visual simulation system, the problems of high cost and subjectivity of manual labeling are solved, and automated truncation ratio labeling is realized, which meets the complex scenario needs of autonomous driving applications.
Patent Information
- Application Number
- CN202411161155.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, truncated proportional annotation is performed manually, resulting in high cost, low efficiency and subjectiveness, making it difficult to adapt to the needs of large-scale data and complex scenarios in autonomous driving applications.
By generating a BEV spatial image in the vehicle vision simulation system, the image acquisition device of the target simulation vehicle is determined in the field of view area, and the projection area of the target object in the image is determined according to the simulation parameters, the truncation ratio is calculated, and automatic labeling is realized .
Get rid of the tedious work of manual annotation, and realize the automatic annotation of the truncation ratio of objects in the image, improve efficiency and accuracy, and adapt to the complex scenario needs in autonomous driving applications.
Smart Images

Figure CN119027533A_ABST
Abstract
Description
Technical Field
[0001] The present specification relates to the field of image processing technology, and in particular to a method, device and electronic device for determining a truncation ratio of an object in an image. Background Art
[0002] In the field of machine learning and computer vision, data annotation is a key step in training high-quality models. In particular, in application areas such as autonomous driving, video surveillance, and image recognition, accurate data annotation is essential for training models to recognize and understand complex scenes. Truncation ratio annotation is an important aspect of data annotation, which involves determining the extent to which target objects (such as vehicles, pedestrians, etc.) are truncated in an image. By annotating the truncation ratios of various target objects appearing in an image, machine learning models can learn to effectively identify target objects even when they are incomplete, thereby improving the robustness of the model in real-world applications. In autonomous driving applications, accurate truncation ratio annotation ensures that the model can correctly handle target objects that are partially visible in the image.
[0003] Using simulation systems combined with large-model data synthesis technology to efficiently synthesize data has been a hot topic in recent years. However, since the simulation system does not provide truncated scale annotation data for synthetic images, users still need to complete the truncated scale annotation work themselves. However, the current annotation method still uses manual annotation, which is not only labor-intensive but also subjective, and is difficult to meet the needs of truncated scale annotation. Summary of the invention
[0004] To overcome the problems existing in the related art, this specification provides a method, a device and an electronic device for determining a truncation ratio of an object in an image.
[0005] According to a first aspect of an embodiment of this specification, a method for determining a truncation ratio of an object in an image is provided, the method being applied to a visual simulation system of a vehicle, comprising:
[0006] Generate BEV space images;
[0007] Determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system, wherein the image to be annotated is generated by simulation by the visual simulation system;
[0008] The truncation ratio of the target object in the image to be labeled is determined based on the image information of the field of view area and the projection area.
[0009] According to a second aspect of an embodiment of this specification, a device for determining a truncation ratio of an object in an image is provided, wherein the device is applied to a visual simulation system of a vehicle, and comprises:
[0010] A BEV space image generation module, used for generating a BEV space image;
[0011] an area determination module, used to determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system, wherein the image to be annotated is generated by simulation by the visual simulation system;
[0012] A truncation ratio determination module is used to determine the truncation ratio of the target object in the image to be labeled based on the image information of the field of view area and the projection area.
[0013] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method described in the first aspect are implemented.
[0014] According to a fourth aspect of the embodiments of this specification, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method described in the first aspect.
[0015] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:
[0016] In the embodiments of this specification, a BEV space image is generated; the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image is determined, and the projection area of the target object that appears around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image is determined according to the simulation parameters of the visual simulation system, and finally the truncation ratio of the target object in the image to be annotated is determined based on the image information of the field of view area and the projection area. It can be seen that this solution gets rid of the tedious work of manually determining the truncation ratio of the object in the image, and realizes the automation of the truncation ratio annotation of the object in the image in the vehicle visual simulation system.
[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.
[0019] Figure 1 It is a schematic diagram of a target simulation vehicle image acquisition process shown in this specification according to an exemplary embodiment.
[0020] Figure 2 The present specification is a flowchart of a method for determining a truncation ratio of an object in an image according to an exemplary embodiment.
[0021] Figure 3 It is a schematic diagram of a BEV space image shown in this specification according to an exemplary embodiment.
[0022] Figure 4 It is a schematic diagram of determining a field of view area according to an exemplary embodiment of this specification.
[0023] Figure 5 It is a schematic diagram of determining a projection area according to an exemplary embodiment of this specification.
[0024] Figure 6 It is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present specification.
[0025] Figure 7 The present specification is a block diagram of a device for determining a truncation ratio of an object in an image according to an exemplary embodiment. DETAILED DESCRIPTION
[0026] The vehicle's visual simulation system can provide simulation data to simulate real-world driving environments, road conditions, and traffic conditions to train and evaluate autonomous driving algorithms. The system can create detailed virtual environments, including urban roads, highways, rural roads, etc. These environments contain various road signs, traffic signals, pedestrians, vehicles, and other objects. Through high-fidelity environmental modeling, the system is able to generate a variety of complex driving scenarios. The system can simulate various sensors used by autonomous vehicles, such as cameras, LiDAR, radar, and ultrasonic sensors. The simulation system can also simulate the behavior of other traffic participants, including other vehicles, pedestrians, bicycles, etc. Through dynamic traffic simulation, the system can generate complex traffic scenarios to help test the performance of autonomous driving algorithms under different traffic conditions.
[0027] The visual simulation system of the vehicle can provide developers with APIs for relevant simulation data, so that developers can customize the simulation data for specific needs. For example, developers are allowed to customize specific traffic scenes, sensor configurations, and vehicle conditions. Although developers can generate simulated image data of the surrounding environment captured by the image acquisition device of the simulated vehicle in the virtual driving scene in the visual simulation system of the vehicle through relevant APIs, it is impossible to automatically generate a labeling file of the truncation ratio of each target object in the simulated image data. Developers are still required to generate a labeling file of the truncation ratio of the target object in the simulated image data separately for each simulated image data.
[0028] The so-called truncation ratio of the target object in the image is used to describe the degree to which the target object is truncated in the image data, and can be expressed in the form of a ratio. For example, due to the limited visual range of the camera, the image data only contains a part of the captured target object, and the remaining part of the target object does not appear in the image data. In this case, the ratio of the area of the part of the target object that appears in the image data to the total area of the target image can be regarded as the truncation ratio of the target object in the image data. For example, if only half of the target object is visible in the image, its truncation ratio is 0.5.
[0029] By annotating the cutoff ratios of various target objects appearing in an image, machine learning models can learn to effectively identify target objects even when they are incomplete, thereby improving the robustness of the model in real-world applications. In autonomous driving applications, accurate cutoff ratio annotation ensures that the model can correctly handle partially visible target objects in the image.
[0030] Existing truncated scale labeling work mainly relies on manual work. Although this method can provide high labeling accuracy, it still has some significant disadvantages. Manual labeling is a costly and inefficient process that requires a lot of human resources and is accompanied by significant labor costs. In addition, the labeling results may be subjective due to the possible differences in personal understanding and standards of the annotators. Long-term repetitive work may also cause annotator fatigue, increase the error rate, and is a time-consuming process, especially for large-scale data, which may take weeks or even months to complete. In addition, manual labeling has poor scalability and is difficult to cope with the growing data demand.
[0031] In response to the above technical problems, this specification provides a method for determining the truncation ratio of an object in an image, which automatically determines the truncation ratio of a target object in simulated image data without manual intervention, thereby improving the efficiency of labeling.
[0032] Next, the embodiments of this specification are described in detail.
[0033] like Figure 1 As shown, it is assumed that in the visual simulation system of the vehicle, it is necessary to perform truncation and proportional annotation on the target object in the image collected by the image acquisition device of the target simulation vehicle 15. The target simulation vehicle 15 can be a test vehicle simulating unmanned driving. The system allows the user to configure a number of virtual image acquisition devices at different positions of the body of the target simulation vehicle 15 to simulate the image acquisition devices in the real world to collect environmental image information around the vehicle. For example, two image acquisition devices can be installed at the front of the target simulation vehicle 15, two image acquisition devices can be installed at the rear, and one image acquisition device can be installed on both sides of the body to simulate the full range of environmental image information around the vehicle. The type of the image acquisition device can be a vehicle-mounted camera, and the user can pre-set the configuration parameters of the image acquisition device on the simulation system according to the needs, such as parameters such as lens focal length and sensor width. The user can pre-set the range of the surrounding environment that the target simulation vehicle 15 needs to detect on the simulation system. For example, the user can set the detection range 17 with the target simulation vehicle 15 as the center, 100 meters before and after the target simulation vehicle 15, and 50 meters to the left and right. Optionally, after determining the detection range 17, the object to be annotated may be any object that appears within the detection range 17, and there is no need to calculate the truncation ratio of the object outside the detection range 17 in the image to be annotated. For example, the object may be limited to the object 12, the object 14, and the object 16 within the detection range 17, but does not include the object 18 outside the detection range 17. Optionally, the user can specify the object type and only annotate a certain type of object with a truncation ratio. For example, only target object types such as pedestrians or trees may be annotated with a truncation ratio. Of course, the user can also annotate all objects that appear within the detection range 17. This specification does not limit the scope of objects for annotating the truncation ratio.
[0034] If the target simulation vehicle 15 is equipped with multiple image acquisition devices, the method for determining the truncation ratio of the object in the image in this specification can be to perform truncation ratio annotation on the object in the image to be annotated acquired by any image acquisition device of the target simulation vehicle 15. For example, at the acquisition moment of the image acquisition device 13 simulating the acquisition of a certain image to be annotated 11, the target objects appearing in the detection range 17 are object 12, object 14 and object 16. Among them, the object 12 partially appears in the viewing angle range of the image acquisition device 13, and the truncation ratio of the object 12 in the image to be annotated 11 can be calculated, and for the objects 14 and 16 that do not appear in the viewing angle range of the image acquisition device, their truncation ratios in the image to be annotated 11 are 0.
[0035] like Figure 2 As shown, Figure 2This is a flow chart of a method for determining a truncation ratio of an object in an image according to an exemplary embodiment of the present specification, wherein the method is applied to a visual simulation system of a vehicle, and comprises steps 201-203:
[0036] Step 201: Generate a BEV space image.
[0037] BEV (Bird's Eye View) spatial image, also known as "bird's eye view", is a method of representing 3D space on a plane, similar to an image taken from an overhead perspective, and is commonly used in fields such as autonomous driving, drone monitoring, and robot navigation. In the field of autonomous driving, BEV spatial images provide an intuitive way to observe the environment around the vehicle to help the vehicle better perceive and understand the surrounding spatial layout.
[0038] For example, Figure 3 As shown, a blank BEV (Bird's Eye View) space image 21 is generated with the target simulated vehicle 15 as the center. For example, first, the size of the BEV space image 21 can be set, for example, the size of the BEV space image 21 can be larger than the detection range 17. Secondly, with the target simulated vehicle as the coordinate origin, an x, y coordinate axis can be established in the BEV space image 21 to assist in representing the relative position information of the object and the vehicle in the BEV image.
[0039] It is understandable that the BEV spatial image 21 can be established only once, and can be reused when determining the truncation ratio of objects in different images to be labeled acquired by the image acquisition device. This reduces the overhead of regenerating the BEV spatial image 21 each time the truncation ratio of objects in different images to be labeled is annotated, thereby improving processing efficiency.
[0040] Step 202: Determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system.
[0041] The field of view area of the image acquisition device of the target simulated vehicle 15 in the BEV space image 21 refers to the area that the image acquisition device can perceive and cover in the BEV space image 21. The image acquisition device of the target simulated vehicle 15 in this embodiment can be any image acquisition device on the target simulated vehicle 15. For example, the image acquisition device can be Figure 1 The image acquisition device 13 of the front of the vehicle. Of course, the image acquisition device can also be Figure 1The method of this embodiment is applicable to any image acquisition device of the target simulation vehicle 15. In addition, the image to be annotated can be generated by the visual simulation system for the image acquisition device through simulation.
[0042] In an illustrated embodiment, the HFOV (Horizontal Field of View, also known as the "horizontal field of view") information of the image acquisition device can be obtained from the simulation parameters of the vehicle's visual simulation system, and the field of view area of the image acquisition device in the BEV space image can be determined based on the HFOV information. Among them, the HFOV information refers to the maximum field of view angle that the image acquisition device can cover on the horizontal plane, which usually depends on parameters such as the focal length of the image acquisition device and the size of the sensor in the image acquisition device. Regarding how to calculate the HFOV information, please refer to the relevant technology for details, and this specification will not go into details here. Since the HFOV information depends on the hardware parameters of the image acquisition device, after the HFOV information is calculated, the HFOV information can be stored in the vehicle's visual simulation system, and can be directly obtained when the HFOV information is needed, without the need to recalculate the HFOV information each time based on the hardware parameters of the image acquisition device.
[0043] After acquiring the HFOV information of the image acquisition device, the HFOV information can be mapped to the BEV space image 21 established in step 201. For example, any image to be annotated acquired by the image acquisition device can be acquired, and the field of view area of the image acquisition device in the BEV space image can be determined based on the HFOV information and the pixel position information of the image to be annotated. Specifically, formula (1) can be used for calculation:
[0044]
[0045] Where HFOV represents the HFOV parameter, p x It represents the horizontal pixel coordinate information of any pixel on the leftmost or rightmost side of the image to be annotated, and w represents the pixel width of the image to be annotated.
[0046] like Figure 4 As shown, taking the image acquisition device 13 as an example, the y-axis is used as the central axis, and the rays are extended at angles θ on both sides of the central axis. Then, the two rays can be closed by the upper edge of the detection range 17, and finally a triangular field of view area 41 is formed.
[0047] Of course, the above is only an exemplary description. In addition to the above embodiments, there are other ways to determine the field of view area 41. For details, please refer to the relevant technology, and this specification will not go into details here.
[0048] It should be noted that the shape and size of the field of view area 41 depend on the hardware parameters and detection range 17 of the image acquisition device 13, and the position of the field of view area 41 depends on the relative position to the target simulated vehicle 15. Therefore, when the hardware parameters, detection range 17 and relative position of the image acquisition device 13 remain unchanged, the field of view area 41 of the BEV spatial image can be determined only for the first time, and for any image 11 to be annotated acquired by the image acquisition device 13, the BEV spatial image 21 with the field of view area 41 drawn thereon can be reused later.
[0049] For the image to be annotated that is acquired by the image acquisition device at the acquisition time of any image to be annotated, that is, the image that needs to be truncated and proportionally annotated, the target objects that appear around the vehicle at the acquisition time of the current image to be annotated can be determined. For example, assuming that at the acquisition time T1 of the image to be annotated, the image acquisition device simulates the acquisition of image P1, and the target objects that appear around the vehicle at this time are A1, A2 and A3, then the projection areas of the target objects A1, A2 and A3 in the BEV space image 21 can be determined; and at the acquisition time T2 of the image to be annotated, the image acquisition device simulates the acquisition of image P2, and the target objects that appear around the vehicle at this time are A4 and A5, then the projection areas of the target objects A4 and A5 in the BEV space image 21 can be determined.
[0050] Since the visual simulation system of the vehicle simulates the real driving environment according to the preset simulation parameters, the relevant information of the target objects appearing around the target simulation vehicle 15 at the time of collecting the image to be annotated can be determined according to the simulation parameters of the visual simulation system. For example, the type of objects, the number of objects, the position of objects, the movement trajectory of objects and other information around the target simulation vehicle 15 can be determined according to the visual simulation parameters.
[0051] In an illustrated embodiment, simulation parameters can be obtained through the interface of the visual simulation system, and the simulation parameters include spatial coordinate information of the target object that appears around the target simulation vehicle at the time of acquisition of the image to be annotated, and the spatial coordinate information can be mapped to the BEV spatial image 21 to form a projection area. For example, for the spatial coordinates (x, y, z) of any pixel point on the edge of the target object, the spatial coordinates can be projected onto the BEV spatial image 21, and the coordinates of the pixel point in the BEV spatial image 21 are (x, y). The projection area is obtained by projecting all the pixel points on the edge of the target object onto the BEV spatial image 21.
[0052] In another illustrated embodiment, the spatial coordinate information may be the spatial coordinate information of the bounding box of the target object, so that for any irregularly shaped object, the spatial coordinate information of the bounding box can be mapped to the BEV spatial image 21 to form a projection area. The shape of the bounding box may be a cube, which can completely contain the target object. By projecting the cube in the three-dimensional space onto the BEV spatial image 21 in the two-dimensional space, the projection area formed by the bounding box in the BEV spatial image 21 can be obtained. For example, for any bounding box, the coordinates of the four vertices on its upper surface are (x1, y1, z1), (x2, y2, z2), (x3, y3, z3) and (x4, y4, z4), respectively. When the target object is mapped to the BEV spatial image 21, the coordinates of the four corners of the projection area formed in the BEV spatial image 21 are (x1, y1), (x2, y2), (x3, y3) and (x4, y4), respectively.
[0053] For example, Figure 5 As shown, according to the processing method of the above embodiment, the image to be annotated that appears at the time of acquisition can be Figure 1 The target objects 12 , 14 , and 16 around the target simulation vehicle 15 are respectively mapped as projection areas 51 , 52 , and 53 on the BEV space image 21 .
[0054] In an illustrated embodiment, the mapping ratio between the image coordinate system in the BEV space image 21 and the physical coordinate system in the real world can be set in the visual simulation system. For example, each pixel in the image corresponds to 0.1m, which means that one pixel unit in the image corresponds to 0.1m in the real world. Assuming that the resolution of the BEV space image is 1000*1000, it corresponds to an actual area of 100m*100m in the real world. Therefore, after determining the mapping ratio between the image coordinate system and the physical coordinate system, the actual physical area of any area in the BEV space image 21 can be converted based on the number of pixels in the area. For example, if the mapping ratio is one pixel corresponds to 0.1m, the area of one pixel is 0.1*0.1, and if the area contains a total of 100 pixels, the pixel area of the area can be converted to 0.1*0.1*100=1 square meter.
[0055] It should be noted that the mapping ratio between the image coordinate system and the physical coordinate system can be freely set according to actual needs. For example, each pixel point in the BEV space image 21 can also be set to correspond to different lengths such as 0.2m, 0.3m, etc.
[0056] In an illustrated embodiment, the ratio of the overlapping area between the projection area and the field of view area of the target object in the BEV space image 21 to the total area of the projection area can be equated to the truncation ratio of the target object in the image to be annotated. Figure 5 As shown, the truncation ratio of the target object 12 in the image to be annotated 11 can be calculated in the following manner:
[0057] In an illustrated embodiment, a field of view pixel set composed of each pixel in the field of view area 41 and a projection pixel set composed of each pixel in the projection area 51 can be determined, and the ratio of the number of pixels included in the intersection of the field of view pixel set and the projection pixel set to the number of pixels included in the projection pixel set is used as the truncation ratio of the target object 12 in the image to be annotated 11. For example, the field of view pixel set x in the field of view area 41 and the projection pixel set y in the projection area 51, then the truncation ratio of the target object 12 in the image to be annotated 11 is (x∩y) / x.
[0058] In another illustrated embodiment, the field of view pixel set composed of each pixel in the field of view area 41, the projection pixel set composed of each pixel in the projection area 51, and the geometric projection area of the projection area 51 are determined, and the ratio of the pixel area represented by the intersection of the field of view pixel set and the projection pixel set to the geometric projection area is used as the truncation ratio of the target object 12 in the image to be annotated 11. For example, assuming that a pixel is set to correspond to 0.1m, and the number of pixels contained in the intersection of the field of view pixel set and the projection pixel set is 100, then the pixel area represented by the intersection is 0.1*0.1*100=1 square meter. Assuming that the length of the projection area 51 is 2m and the width is 1m, the geometric projection area of the projection area is 2*1=2 meters. The truncation ratio of the target object 12 in the image to be annotated 11 is 1 / 2.
[0059] In another illustrated embodiment, the geometric overlap area of the overlapping portion of the field of view area 41 and the projection area 51 and the geometric projection area of the projection area 51 are determined, and the ratio of the geometric overlap area to the geometric projection area is used as the truncation ratio of the target object 12 in the image to be annotated 11. The geometric overlap area can be calculated by a grid method or an integral method, and then the geometric projection area of the projection area 51 is calculated by the length and width of the projection area 51.
[0060] For the projection area 52 and the projection area 53 that do not overlap with the field of view area 41 in the BEV space image 21 , it means that the truncation ratio of the target object 14 and the target object 16 in the image to be labeled 11 is 0.
[0061] In an illustrated embodiment, after determining the truncation ratio of the target object in the image to be annotated 11, a labeling file of the image can be generated. For example, the labeling file can include, but is not limited to, recording the simulated vehicle number, the image acquisition device number, the image number, the target object number, and the truncation ratio of the target object. For example, for the image to be annotated 11 captured by the image acquisition device 13 of the target simulated vehicle 15 at a certain capture time of the image to be annotated, the truncation ratios of the target objects 12, 14, and 16 can be recorded in the labeling file.
[0062] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a device and a terminal to which it is applied.
[0063] like Figure 6 As shown, Figure 6 It is a structural diagram of an electronic device shown in this specification according to an exemplary embodiment. At the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, a memory 608 and a non-volatile memory 610, and of course may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 602 reading the corresponding computer program from the non-volatile memory 610 into the memory 608 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic module, but can also be hardware or logic devices.
[0064] like Figure 7 As shown, Figure 7 is a block diagram of a device for determining a truncation ratio of an object in an image according to an exemplary embodiment of the present specification. The device can be applied to Figure 6 In the electronic device 600 shown in the figure, the technical solution of this specification is implemented. The device includes:
[0065] A BEV space image generation module 702, used to generate a BEV space image;
[0066] An area determination module 704 is used to determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system, wherein the image to be annotated is generated by simulation by the visual simulation system;
[0067] The truncation ratio determination module 706 is used to determine the truncation ratio of the target object in the image to be labeled based on the image information of the field of view area and the projection area.
[0068] Optionally, the area determination module 704 is specifically configured to obtain HFOV information of the image acquisition device, and determine the field of view area of the image acquisition device in the BEV space image based on the HFOV information.
[0069] Optionally, the area determination module 704 is specifically used to obtain an image corresponding to the image acquisition device; based on the HFOV information and the pixel position information of the image to be annotated, determine the field of view area of the image acquisition device in the BEV space image.
[0070] Optionally, the area determination module 704 is specifically used to obtain the simulation parameters through the interface of the visual simulation system, and the simulation parameters include spatial coordinate information of target objects appearing around the target simulation vehicle at the time of acquisition of the image to be annotated; and map the spatial coordinate information into the BEV spatial image to form the projection area.
[0071] Optionally, the spatial coordinate information includes spatial coordinate information of a bounding box of the target object.
[0072] Optionally, the truncation ratio determination module 706 is specifically used to determine a field of view pixel point set composed of each pixel point in the field of view area and a projection pixel point set composed of each pixel point in the projection area, and take the ratio of the number of pixels included in the intersection of the field of view pixel point set and the projection pixel point set to the number of pixels included in the projection pixel point set as the truncation ratio of the target object in the image to be labeled; or,
[0073] Determine a field of view pixel point set composed of each pixel point in the field of view area, a projection pixel point set composed of each pixel point in the projection area, and a geometric projection area of the projection area, and take the ratio of the pixel point area represented by the intersection of the field of view pixel point set and the projection pixel point set to the geometric projection area as the truncation ratio of the target object in the image to be labeled.
[0074] Optionally, the device further comprises a labeling file generating module 708, configured to generate a labeling file corresponding to the image, wherein the labeling file at least comprises a truncation ratio of a target object in the image to be labeled.
[0075] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.
[0076] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.
[0077] The present specification also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of any of the aforementioned methods for determining the truncation ratio of an object in an image provided in the present application are implemented.
[0078] Specifically, computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks.
Claims
1. A method for determining a truncation ratio of an object in an image, characterized in that: The method is applied to a visual simulation system of a vehicle, comprising: Generate BEV space images; Determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system, wherein the image to be annotated is generated by simulation by the visual simulation system; The truncation ratio of the target object in the image to be labeled is determined based on the image information of the field of view area and the projection area.
2. The method according to claim 1, characterized in that The step of determining the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image includes: HFOV information of the image acquisition device is acquired, and a field of view area of the image acquisition device in the BEV space image is determined based on the HFOV information.
3. The method according to claim 2, characterized in that The method of determining the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image based on the HFOV information includes: Acquire an image to be annotated corresponding to the image acquisition device; Based on the HFOV information and the pixel position information of the image to be annotated, the field of view area of the image acquisition device in the BEV space image is determined.
4. The method according to claim 1, characterized in that: The step of determining, according to the simulation parameters of the visual simulation system, a projection area in the BEV space image of a target object that appears around the target simulation vehicle at the time of collecting the image to be annotated, comprises: Acquiring the simulation parameters through the interface of the visual simulation system, wherein the simulation parameters include spatial coordinate information of target objects that appear around the target simulation vehicle at the time of acquisition of the image to be annotated; The spatial coordinate information is mapped into the BEV spatial image to form the projection area.
5. The method according to claim 4, characterized in that The spatial coordinate information includes spatial coordinate information of a bounding box of the target object.
6. The method according to claim 1, characterized in that The determining the truncation ratio of the target object in the image to be labeled based on the image information of the field of view area and the projection area includes: Determine a field of view pixel set composed of each pixel in the field of view area and a projection pixel set composed of each pixel in the projection area, and use the ratio of the number of pixels included in the intersection of the field of view pixel set and the projection pixel set to the number of pixels included in the projection pixel set as the truncation ratio of the target object in the image to be labeled; or Determine a field of view pixel point set composed of each pixel point in the field of view area, a projection pixel point set composed of each pixel point in the projection area, and a geometric projection area of the projection area, and take the ratio of the pixel point area represented by the intersection of the field of view pixel point set and the projection pixel point set to the geometric projection area as the truncation ratio of the target object in the image to be labeled.
7. The method according to claim 1, characterized in that The method further comprises: Generate a labeling file for the image to be labeled, wherein the labeling file at least includes a truncation ratio of the target object in the image to be labeled.
8. A device for determining the truncation ratio of an object in an image, characterized in that The device is applied to a visual simulation system of a vehicle, comprising: A BEV space image generation module, used for generating a BEV space image; an area determination module, used to determine the field of view area of the image acquisition device of the target simulated vehicle in the BEV space image, and determine the projection area of the target object appearing around the target simulated vehicle at the time of acquisition of the image to be annotated in the BEV space image according to the simulation parameters of the visual simulation system, wherein the image to be annotated is generated by simulation by the visual simulation system; A truncation ratio determination module is used to determine the truncation ratio of the target object in the image to be labeled based on the image information of the field of view area and the projection area.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
