Performance evaluation method and device for natural scene object detection system

Abstract

The present invention provides a performance evaluation method and device for a natural scene target detection system, the method comprising: fusing a target image with a plurality of natural scenes to obtain a target image fused with the natural scenes; segmenting the target image fused with the natural scenes to obtain the object category of each object in the target image; inputting the target image fused with the natural scenes into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system; calculating an evaluation index value based on the object category of each object in the target image and the target object detection result; and evaluating the performance of the natural scene target detection system according to the evaluation index value. The present invention can evaluate the performance of a natural scene target detection system with low cost and high accuracy.

Description

Technical Field

[0001] The present invention relates to the field of detection technology, and in particular to a performance evaluation method and device for a natural scene target detection system. Background Art

[0002] In recent years, machine learning technology centered on deep learning has attracted widespread attention. For example, autonomous driving has gradually become possible. A key link in autonomous driving is the effective cognition of traffic scenes, especially how to improve the resolution efficiency of sensors under severe weather conditions, and the accurate identification and detection of the feasible domain of roads. These factors directly determine the safety performance of autonomous vehicles. However, the influence of external interference factors on the reasoning of AI systems is difficult to control and evaluate. Studies have shown that rain and fog will seriously affect the accuracy of automatic detection systems for pedestrians, cyclists and other road users. Challenging weather conditions in the future will be a key factor affecting the accuracy of artificial intelligence reasoning in machine vision-related tasks. Therefore, it is particularly important to evaluate the performance of natural scene object detection systems such as the above-mentioned AI. However, there is currently a lack of a low-cost and high-accuracy performance evaluation method for natural scene object detection systems. Summary of the invention

[0003] The embodiment of the present invention provides a performance evaluation method of a natural scene object detection system, which is used to evaluate the performance of the natural scene object detection system with low cost and high accuracy. The method includes:

[0004] Fusing the target image with multiple natural scenes to obtain a target image fused with the natural scenes;

[0005] Segmenting the target image of the fused natural scene to obtain an object category of each object in the target image;

[0006] Inputting the target image fused with the natural scene into the natural scene target detection system to obtain a second target object detection result output by the natural scene target detection system;

[0007] Calculate an evaluation index value based on the object category of each object in the target image and the target object detection result;

[0008] The performance of the natural scene object detection system is evaluated according to the evaluation index value.

[0009] The embodiment of the present invention provides a performance evaluation device for a natural scene object detection system, which is used to evaluate the performance of the natural scene object detection system, has low cost and high accuracy, and includes:

[0010] A fusion module, used for fusing the target image with multiple natural scenes to obtain a target image fused with the natural scenes;

[0011] A segmentation module, used to segment the target image of the fused natural scene to obtain an object category of each object in the target image;

[0012] A detection module is used to input the target image fused with the natural scene into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system;

[0013] An evaluation index value calculation module, used to calculate an evaluation index value based on the object category of each object in the target image and the target object detection result;

[0014] The performance evaluation module is used to evaluate the performance of the natural scene object detection system according to the evaluation index value.

[0015] An embodiment of the present invention further proposes a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned performance evaluation method of the natural scene target detection system when executing the computer program.

[0016] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned performance evaluation method for the natural scene object detection system.

[0017] In an embodiment of the present invention, a target image is fused with a plurality of natural scenes to obtain a target image of the fused natural scene; the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target image of the fused natural scene is input into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system; an evaluation index value is calculated based on the object category of each object in the target image and the target object detection result; and the performance of the natural scene target detection system is evaluated according to the evaluation index value. In the above process, the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target object detection result is obtained by inputting the target image of the fused natural scene into the natural scene target detection system, which is used for comparison with the object category of each object in the target image that has been obtained, and the evaluation index value is calculated, so that the performance of the natural scene target detection system can be accurately and efficiently evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 is a flow chart of a performance evaluation method for a natural scene object detection system according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a performance evaluation method for a natural scene object detection system according to an embodiment of the present invention;

[0021] Figure 3 Schematic diagram of the equation for fog concentration in an embodiment of the present invention;

[0022] Figure 4 Schematic diagram of a target image fused with a fog scene in an embodiment of the present invention;

[0023] Figure 5 A flowchart of training a neural network model in an embodiment of the present invention;

[0024] Figure 6 Schematic diagram of a performance evaluation device for a natural scene object detection system according to an embodiment of the present invention;

[0025] Figure 7 Schematic diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0027] In the description of this specification, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps is not limited and can be appropriately adjusted as needed.

[0028] Figure 1 FIG. 1 is a flow chart of a performance evaluation method for a natural scene object detection system according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0029] Step 101, fusing a target image with multiple natural scenes to obtain a target image fused with the natural scenes;

[0030] Step 102, segmenting the target image of the fused natural scene to obtain the object category of each object in the target image;

[0031] Step 103, inputting the target image fused with the natural scene into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system;

[0032] Step 104, calculating an evaluation index value based on the object category of each object in the target image and the target object detection result;

[0033] Step 105: Evaluate the performance of the natural scene object detection system according to the evaluation index value.

[0034] In the method proposed in the embodiment of the present invention, the target image is fused with multiple natural scenes to obtain a target image of the fused natural scene; the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target image of the fused natural scene is input into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system; an evaluation index value is calculated based on the object category of each object in the target image and the target object detection result; and the performance of the natural scene target detection system is evaluated according to the evaluation index value. In the above process, the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target object detection result is obtained by inputting the target image of the fused natural scene into the natural scene target detection system, which is used for comparison with the object category of each object in the target image that has been obtained, and the evaluation index value is calculated, so that the performance of the natural scene target detection system can be accurately and efficiently evaluated.

[0035] In specific implementation, the embodiment of the present invention integrates images with natural scenes of various weather factors (fog, rain, lightning, snow, etc.) to achieve controllable environmental interference factors, construct test materials that can quantify interference factors, and evaluate the performance of the natural scene target detection system. Figure 2: is a schematic diagram of a performance evaluation method for a natural scene target detection system in an embodiment of the present invention, that is, a target image is fused with a natural scene, for example, a fog scene formed by an OpenGL atomization effect and a rain scene formed by an OpenGL raining effect, to form a target image of a fused fog scene and a target image of a fused rain scene, respectively, and then input them into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system, and combine it with the object category of each object in the target image obtained by segmentation to calculate an evaluation index value. Figure 2 Several examples of evaluation indicators are listed in the table, such as object prediction accuracy, memory usage, and execution time. The following is a detailed introduction.

[0036] In one embodiment, the natural scene includes a fog scene;

[0037] The target image is fused with a variety of natural scenes to obtain a target image fused with natural scenes, including:

[0038] Constructing a fog scene according to a preset fog mode, color and concentration, wherein the fog mode includes linear fog, exponential fog or exponential square fog;

[0039] The target image is fused with the fog scene to obtain a target image fused with the fog scene.

[0040] In the above embodiment, OpenGL can be used to construct the fog scene. OpenGL is the most widely accepted 2D / 3D graphics API in the industry. Since its birth, it has spawned thousands of excellent applications on various computer platforms and devices. OpenGL can create various weather scenes. The first is the fog effect. The implementation principle is to calculate a mixing factor based on the distance of the fog from the observation point, and then use it to mix the fog color with the target color to obtain the effect. In the OpenGL scene, there are three fog equations of fog modes to create the fog effect, namely linear fog, exponential fog and exponential squared fog. Their realism: exponential fog < exponential squared fog < linear fog.

[0041] Figure 3 Schematic diagram of the fog concentration equation in the embodiment of the present invention, it can be observed that the trends of the fog concentration equations of the three fog modes are: Figure 3 In the example, GL_LINEAR is linear fog, GL_EXP is exponential fog, GL_EXP2 is exponential square fog, fogStart is the minimum distance from the original color to the observer, and fogEnd is the maximum distance from the original color to the observer.

[0042] Linear fog is the most realistic fog equation. It is a linear interpolation function that can mix the fog color with the original color according to the distance between the observation point and the scene. The linear fog equation requires two parameters. The first parameter is fogStart, which is called the minimum distance. It represents the location information where the fog starts to mix. The second parameter is fogEnd, which is the maximum distance from the observer to the fog color covering the original color. The function equation is as follows:

[0043]

[0044] c represents the distance from the rendered fragment to the viewing plane; f is the fog factor, which should be between 0 and 1 to ensure that the original color and the fog color are mixed correctly.

[0045] The target image equation for fused fog scene is as follows:

[0046] finalColor=(1-f)×fogColor+f×originalColor (2)

[0047] Among them, finalColor is the color of the target image fused with the fog scene, originalColor is the original color of the target image, and fogColor is the fog color.

[0048] Exponential fog is a simplified model of real-world phenomena. Since particles in the air absorb and scatter light, the light intensity is attenuated over the entire viewing distance. Therefore, each time the light passes through a particle, its intensity is absorbed by a small attenuation factor, which causes the light intensity to decrease exponentially with increasing distance from the observer. In the simplified model, this attenuation factor is called fog density and it is uniform. The functional equation of exponential fog is as follows:

[0049]

[0050] Where d is the distance and b is the attenuation factor. The target image equation for the fused fog scene is the same as equation (2).

[0051] The only difference between Exponential Squared Fog and Exponential Fog is that the distance and attenuation factor are squared. In addition to incident light, this equation also takes into account reflected light, which will have clearer visibility than the previous two modes at a small range from the observer, and then the fog density increases rapidly as the distance increases. The function equation is as follows:

[0052]

[0053] Where d is the distance and b is the attenuation factor. The target image equation for the fused fog scene is the same as equation (2). Figure 4This is a schematic diagram of a target image for a fused fog scene in an embodiment of the present invention. The target image can be loaded through OpenGL and converted into a texture, and then 2D texture mapping is enabled, the fog mode (linear fog, exponential fog, or exponential square fog) is selected, the color and density of the fog are set, and the fog effect is enabled. The density of the fog is modified to generate a target image that fuses fogs of different densities.

[0054] In one embodiment, the natural scene includes a rain scene;

[0055] The target image is fused with a variety of natural scenes to obtain a target image fused with natural scenes, including:

[0056] Constructing a rain scene according to preset properties of rain particles, wherein the properties of the particles include at least one of density, size, speed and movement direction;

[0057] The target image is fused with the rain scene to obtain a target image fused with the rain scene.

[0058] In the above embodiment, the effect of rain can also be simulated by using a particle system in OpenGL. The particle system is composed of many simple voxels, which are called particles. They have multiple properties, such as size, shape, color, movement speed, movement direction, life cycle, transparency, etc. Raindrops are represented by texture maps using very small rectangles. During initialization, many objects need to be created and given initial properties, and then run in a loop. The visual characteristics of rainfall require particles to have the following properties: density, size, speed and movement direction. When constructing a rain scene through OpenGL, after determining the properties of rain particles, the particles are initialized, the properties of the particles are updated, and then the particles are drawn. The particles are set to multiple small rectangles, and the two-dimensional texture image of the raindrops is mapped to the rectangular surface of the particles using OpenGL's texture fusion. The background of the two-dimensional texture image of the raindrops is black and the texture is white.

[0059] In one embodiment, after segmenting the target image of the fused natural scene to obtain the object category of each object in the target image, the method further includes:

[0060] Adding a standard label to the object category of each object in the target image obtained after segmentation;

[0061] Based on the object category of each object in the target image and the target object detection results, the evaluation index values ​​are calculated, including:

[0062] Obtain the actual label corresponding to the object category of each object in the target object detection result;

[0063] Based on the standard label and the actual label, obtaining objects whose object categories are correctly predicted and objects whose object categories are incorrectly predicted in the target image;

[0064] The evaluation index value is calculated based on the objects whose object categories are correctly predicted and the objects whose object categories are incorrectly predicted in the target image.

[0065] In the above embodiment, the segmentation of the target image of the fused natural scene is actually semantic segmentation. The purpose of adding standard labels is to facilitate the comparison with the actual labels in the subsequent target object detection results. It is efficient, and the objects with standard labels and the original target images can be stored. In addition, the existing natural scene target detection systems are often obtained by training neural network models. The trained neural network model itself has actual labels, so it is very convenient to compare with standard labels.

[0066] In one embodiment, the evaluation index includes at least one of object prediction accuracy, memory usage, and execution time.

[0067] In the above embodiment, the object prediction accuracy mainly depends on the values ​​of MPA (mean prediction accuracy) and MIoU (mean intersection over union). The larger the values ​​of MPA and MIoU, the better the effect of the semantic segmentation algorithm. In one embodiment, the object prediction accuracy includes the mean intersection over union and the mean prediction accuracy.

[0068] In one embodiment, the average intersection-over-intersection ratio is calculated using the following formula:

[0069]

[0070] Among them, MIoU is the mean intersection over union ratio;

[0071] k+1 (0 to k categories) is the total number of categories of objects; p ij is the object of object category i that is predicted to be of object category j; p ii is the object whose object category is correctly predicted; p ji For objects of object category j, they are predicted to be of object category i;

[0072] The average prediction accuracy is calculated using the following formula:

[0073]

[0074] Among them, MPA is the mean prediction accuracy.

[0075] Memory usage is used to evaluate the memory usage of the natural scene object detection system. Although memory can be expanded at any time in most scenarios, memory is also very precious on some embedded devices and cannot be consumed without limit. Therefore, the system's memory consumption is also an evaluation indicator. The less memory is used, the higher the system performance.

[0076] The execution time is used to evaluate the time required for a natural scene object detection system to identify a target image. It can help subsequent researchers evaluate the effectiveness of the method and test the fastest execution method under the same environment. The shorter the execution time, the higher the performance of the system.

[0077] Based on the above embodiments, a detailed flow chart of a performance evaluation method for a natural scene object detection system is given below. Figure 5 The following is a detailed flow chart of a performance evaluation method for a natural scene object detection system according to an embodiment of the present invention, including:

[0078] Step 501, fusing a target image with multiple natural scenes to obtain a target image fused with the natural scenes;

[0079] Step 502, segmenting the target image of the fused natural scene to obtain the object category of each object in the target image;

[0080] Step 503, adding a standard label to the object category of each object in the target image obtained after segmentation;

[0081] Step 504, inputting the target image fused with the natural scene into a natural scene target detection system to obtain a target object detection result output by the natural scene target detection system;

[0082] Step 505, obtaining the actual label corresponding to the object category of each object in the target object detection result;

[0083] Step 506, based on the standard label and the actual label, obtaining objects whose object categories are correctly predicted and objects whose object categories are incorrectly predicted in the target image;

[0084] Step 507, calculating an evaluation index value according to the objects whose object categories are correctly predicted and the objects whose object categories are incorrectly predicted in the target image;

[0085] Step 508: Evaluate the performance of the natural scene object detection system according to the evaluation index value.

[0086] The above is only one embodiment, and there may be other detailed processes, and related variations should all fall within the protection scope of the present invention.

[0087] In summary, in the method proposed in the embodiment of the present invention, the target image is fused with multiple natural scenes to obtain a target image of the fused natural scene; the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target image of the fused natural scene is input into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system; based on the object category of each object in the target image and the target object detection result, an evaluation index value is calculated; and according to the evaluation index value, the performance of the natural scene target detection system is evaluated. In the above process, the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; by inputting the target image of the fused natural scene into the natural scene target detection system to obtain the target object detection result, which is used for comparison with the object category of each object in the target image that has been obtained, the evaluation index value is calculated, and the performance of the natural scene target detection system can be accurately and efficiently evaluated.

[0088] The embodiment of the present invention further proposes a performance evaluation device for a natural scene object detection system, the principle of which is similar to the performance evaluation method for a natural scene object detection system, and will not be described in detail here.

[0089] Figure 6 FIG. 1 is a schematic diagram of a performance evaluation device for a natural scene object detection system according to an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0090] A fusion module 601 is used to fuse the target image with multiple natural scenes to obtain a target image fused with the natural scenes;

[0091] A segmentation module 602 is used to segment the target image of the fused natural scene to obtain an object category of each object in the target image;

[0092] A detection module 603 is used to input the target image fused with the natural scene into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system;

[0093] An evaluation index value calculation module 604, used to calculate an evaluation index value based on the object category of each object in the target image and the target object detection result;

[0094] The performance evaluation module 605 is used to evaluate the performance of the natural scene object detection system according to the evaluation index value.

[0095] In one embodiment, the natural scene includes a fog scene;

[0096] The fusion module 601 is specifically used for:

[0097] Constructing a fog scene according to a preset fog mode, color and concentration, wherein the fog mode includes linear fog, exponential fog or exponential square fog;

[0098] The target image is fused with the fog scene to obtain a target image fused with the fog scene.

[0099] In one embodiment, the natural scene includes a rain scene;

[0100] The fusion module 601 is specifically used for:

[0101] Constructing a rain scene according to preset properties of rain particles, wherein the properties of the particles include at least one of density, size, speed and movement direction;

[0102] The target image is fused with the rain scene to obtain a target image fused with the rain scene.

[0103] In one embodiment, the segmentation module 602 is further configured to:

[0104] Adding a standard label to the object category of each object in the target image obtained after segmentation;

[0105] The evaluation index value calculation module is specifically used for:

[0106] Obtain the actual label corresponding to the object category of each object in the target object detection result;

[0107] Based on the standard label and the actual label, obtaining objects whose object categories are correctly predicted and objects whose object categories are incorrectly predicted in the target image;

[0108] The evaluation index value is calculated based on the objects whose object categories are correctly predicted and the objects whose object categories are incorrectly predicted in the target image.

[0109] In one embodiment, the evaluation index includes at least one of object prediction accuracy, memory usage, and execution time.

[0110] In one embodiment, the object prediction accuracy includes average intersection-over-union ratio and average prediction precision.

[0111] In one embodiment, the average intersection-over-intersection ratio is calculated using the following formula:

[0112]

[0113] Among them, MIoU is the mean intersection over union ratio;

[0114] k+1 is the total number of object categories; p ij is the object of object category i that is predicted to be of object category j; p ii is the object whose object category is correctly predicted; p jiFor objects of object category j, they are predicted to be of object category i;

[0115] The average prediction accuracy is calculated using the following formula:

[0116]

[0117] Among them, MPA is the mean prediction accuracy.

[0118] In summary, in the device proposed in the embodiment of the present invention, the target image is fused with multiple natural scenes to obtain a target image of the fused natural scene; the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target image of the fused natural scene is input into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system; based on the object category of each object in the target image and the target object detection result, an evaluation index value is calculated; and according to the evaluation index value, the performance of the natural scene target detection system is evaluated. In the above process, the target image of the fused natural scene is segmented to obtain the object category of each object in the target image; the target object detection result is obtained by inputting the target image of the fused natural scene into the natural scene target detection system, which is used for comparison with the object category of each object in the target image that has been obtained, and the evaluation index value is calculated, so that the performance of the natural scene target detection system can be accurately and efficiently evaluated.

[0119] An embodiment of the present application also provides a computer device, Figure 7 Schematic diagram of a computer device in an embodiment of the present invention. The computer device can implement all steps in the performance evaluation method of the natural scene object detection system in the above embodiment. The electronic device specifically includes the following contents:

[0120] Processor (processor) 701, memory (memory) 702, communication interface (CommunicationsInterface) 703 and bus 704;

[0121] The processor 701, the memory 702, and the communication interface 703 communicate with each other through the bus 704; the communication interface 703 is used to realize information transmission between relevant devices such as the server-side device, the detection device, and the user-side device;

[0122] The processor 701 is used to call the computer program in the memory 702, and when the processor executes the computer program, all the steps in the performance evaluation method of the natural scene object detection system in the above embodiment are implemented.

[0123] An embodiment of the present application also provides a computer-readable storage medium that can implement all steps in the performance evaluation method of the natural scene target detection system in the above embodiment. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps in the performance evaluation method of the natural scene target detection system in the above embodiment.

[0124] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A performance evaluation method for a natural scene object detection system, characterized in that: include: Fusing the target image with multiple natural scenes to obtain a target image fused with the natural scenes; Segmenting the target image of the fused natural scene to obtain an object category of each object in the target image; Inputting the target image fused with the natural scene into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system; Calculate an evaluation index value based on the object category of each object in the target image and the target object detection result; The performance of the natural scene object detection system is evaluated according to the evaluation index value.

2. The performance evaluation method of the natural scene object detection system according to claim 1, characterized in that: The natural scene includes a fog scene; The target image is fused with a variety of natural scenes to obtain a target image fused with natural scenes, including: Constructing a fog scene according to a preset fog mode, color and concentration, wherein the fog mode includes linear fog, exponential fog or exponential square fog; The target image is fused with the fog scene to obtain a target image fused with the fog scene.

3. The performance evaluation method of the natural scene object detection system according to claim 1, characterized in that: The natural scene includes a rain scene; The target image is fused with a variety of natural scenes to obtain a target image fused with natural scenes, including: Constructing a rain scene according to preset properties of rain particles, wherein the properties of the particles include at least one of density, size, speed and movement direction; The target image is fused with the rain scene to obtain a target image fused with the rain scene.

4. The performance evaluation method of the natural scene object detection system according to claim 1, characterized in that: After segmenting the target image of the fused natural scene to obtain the object category of each object in the target image, the method further includes: Adding a standard label to the object category of each object in the target image obtained after segmentation; Based on the object category of each object in the target image and the target object detection results, the evaluation index values ​​are calculated, including: Obtain the actual label corresponding to the object category of each object in the target object detection result; Based on the standard label and the actual label, obtaining objects whose object categories are correctly predicted and objects whose object categories are incorrectly predicted in the target image; The evaluation index value is calculated based on the objects whose object categories are correctly predicted and the objects whose object categories are incorrectly predicted in the target image.

5. The performance evaluation method of the natural scene object detection system according to claim 4, characterized in that: The evaluation index includes at least one of object prediction accuracy, memory occupancy, and execution time.

6. The performance evaluation method of the natural scene object detection system according to claim 5, characterized in that: The object prediction accuracy includes average intersection-over-union ratio and average prediction accuracy.

7. The performance evaluation method of the natural scene object detection system according to claim 6, characterized in that: The average intersection-over-intersection ratio is calculated using the following formula: Among them, MIoU is the mean intersection over union ratio; k+1 is the total number of object categories; p ij is the object of object category i that is predicted to be of object category j; p ii is the object whose object category is correctly predicted; p ji For objects of object category j, they are predicted to be of object category i; The average prediction accuracy is calculated using the following formula: Among them, MPA is the mean prediction accuracy.

8. A performance evaluation device for a natural scene object detection system, characterized in that: include: A fusion module, used for fusing the target image with multiple natural scenes to obtain a target image fused with the natural scenes; A segmentation module, used to segment the target image of the fused natural scene to obtain an object category of each object in the target image; A detection module is used to input the target image fused with the natural scene into the natural scene target detection system to obtain the target object detection result output by the natural scene target detection system; An evaluation index value calculation module, used to calculate an evaluation index value based on the object category of each object in the target image and the target object detection result; The performance evaluation module is used to evaluate the performance of the natural scene object detection system according to the evaluation index value.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 7.

Images