Method and device for determining model performance, electronic equipment and readable storage medium

Abstract

The present disclosure relates to a model performance determination method and device, electronic equipment and readable storage medium, the method comprising: obtaining a location of a target vehicle, dividing a plurality of attention regions in a to-be-detected image according to the location of the target vehicle, the attention region being a region containing different entities in the to-be-detected image, using a plurality of correlation models to respectively perform target detection on corresponding adjusted attention regions in the to-be-detected image to obtain a plurality of first detection results, one attention region corresponding to one correlation model, and determining the performance of the plurality of correlation models according to the first detection results; screening out the attention regions according to the location of the target vehicle and a task scene of the target vehicle, and then using the corresponding correlation models to identify the screened attention regions to obtain an identification result, the identification result fused with the location and the task scene of the target vehicle being able to more accurately reflect the real performance of the correlation models.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, and particularly relates to a model performance determination method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] In related technologies, accuracy is the gold standard for measuring models in computer vision. One reason why deep learning models widely use this indicator is that they are largely task-agnostic, and users usually strive for zero errors or as low error rates as possible in detection results.

[0003] However, in vehicle detection of unmanned vehicles, using accuracy as a measurement indicator for vehicle detection models has certain disadvantages. This measurement indicator treats all incorrect detections equally regardless of the task or scenario. For example, in some specific scenarios, the missed detection or incorrect detection of some unimportant areas has no impact on the functional use of the model, but greatly reduces the accuracy of the model. False high accuracy also causes the correct detection of unimportant areas to cover up the missed detection of key areas, resulting in inaccurate evaluation results of the model. SUMMARY

[0004] To overcome the problems in related technologies, the present disclosure provides a model performance determination method and device, electronic equipment and readable storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a model performance determination method is provided, including: obtaining a position of a target vehicle; dividing a plurality of attention regions in a to-be-detected image according to the position of the target vehicle; the plurality of attention regions are regions containing different entities in the to-be-detected image; performing target detection on corresponding adjusted attention regions in the to-be-detected image by using a plurality of associated models to obtain a plurality of first detection results; one attention region corresponds to one associated model; and determining the performance of the plurality of associated models according to the plurality of first detection results.

[0006] Optionally, the adjustment manner of the attention region includes: adjusting the attention region by using the associated model corresponding to the attention region to obtain a plurality of adjusted attention regions.

[0007] Optionally, the step of obtaining the position of the target vehicle includes: obtaining the to-be-detected image according to a camera and / or a laser radar; and positioning the position of the target vehicle according to the to-be-detected image.

[0008] Optionally, the plurality of attention regions include a traffic light attention region, a road sign attention region, a vehicle / pedestrian detection attention region, an obstacle attention region, a traffic sign attention region, and / or a trajectory prediction attention region; and the step of dividing the plurality of attention regions in the to-be-detected image according to the position of the target vehicle includes: positioning the positions of the traffic light attention region, the road sign attention region, the vehicle / pedestrian detection attention region, the obstacle attention region, the traffic sign attention region, and / or the trajectory prediction attention region according to the position of the target vehicle.

[0009] Optionally, the step of adjusting the attention region by using the associated model corresponding to the attention region to obtain a plurality of adjusted attention regions includes: identifying the to-be-detected image by using the associated model corresponding to each of the plurality of attention regions to obtain a plurality of second detection results; the plurality of second detection results include position information of the plurality of attention regions in the to-be-detected image; and adjusting the plurality of attention regions according to the plurality of second detection results.

[0010] Optionally, the step of determining the performance of each of the plurality of associated models according to the plurality of first detection results includes: determining the performance of the associated model according to a corresponding first detection result; the corresponding first detection result is obtained by performing target detection on the corresponding attention region in the to-be-detected image by using the associated model; and the corresponding first detection result includes an accuracy rate and a recall rate.

[0011] According to a second aspect of the embodiments of the present disclosure, a model performance determination apparatus is provided, including: an acquisition module configured to acquire a position of a target vehicle; a processing module configured to divide a plurality of attention regions in a to-be-detected image according to the position of the target vehicle; the plurality of attention regions are regions in the to-be-detected image containing different entities; a detection module configured to perform target detection on corresponding adjusted attention regions in the to-be-detected image by using a plurality of associated models to obtain a plurality of first detection results; one of the attention regions corresponds to one of the associated models; and the processing module is further configured to determine the performance of each of the plurality of associated models according to the plurality of first detection results.

[0012] Optionally, the processing module is further configured to adjust the attention region by using the associated model corresponding to the attention region to obtain a plurality of adjusted attention regions.

[0013] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; and wherein the processor is configured to execute the executable instructions to implement the steps of the model performance determination method described above.

[0014] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer program instructions. The computer program instructions are executed by a processor to implement the steps of the method for determining model performance provided by the first aspect of the present disclosure.

[0015] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects: the position of the target vehicle can be obtained, the plurality of attention regions in the to-be-detected image are divided according to the position of the target vehicle, the attention region is a region containing different entities in the to-be-detected image, the plurality of first detection results are obtained by using the plurality of correlation models to respectively perform target detection on the corresponding and adjusted attention regions in the to-be-detected image, one attention region corresponds to one correlation model; the attention region is filtered according to the position of the target vehicle and the task scene of the target vehicle, and then the filtered attention region is identified by using the corresponding correlation model to obtain an identification result, and the identification result fused with the position of the target vehicle and the task scene can more accurately reflect the real performance of the correlation model.

[0016] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] Figure 1 is a structural schematic diagram of a computer system according to an exemplary embodiment of the present disclosure.

[0019] Figure 2 is a scene schematic diagram of the driving of the target vehicle according to an exemplary embodiment of the present disclosure.

[0020] Figure 3 is a flowchart of a method for determining model performance according to an exemplary embodiment of the present disclosure.

[0021] Figure 4 is a flowchart of a method for determining model performance according to an exemplary embodiment of the present disclosure.

[0022] Figure 5 is a block diagram of a device for determining model performance according to an exemplary embodiment of the present disclosure.

[0023] Figure 6 is a block diagram of a device for determining model performance according to an exemplary embodiment of the present disclosure.

[0024] Figure 7is a block diagram of an apparatus for determination of model performance according to an exemplary embodiment.

[0025] LEGEND

[0026] 120 - terminal; 140 - server; 10 - target vehicle; 20 - determination of model performance apparatus; 201 - acquisition module; 202 - processing module; 204 - detection module; 800 - apparatus; 802 - processing component; 804 - memory; 806 - power component; 808 - multimedia component; 810 - audio component; 812 - I / O interface; 814 - sensor component; 816 - communication component. DETAILED DESCRIPTION

[0027] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same drawings reference numbers are used to denote like or similar elements. The embodiments described in the following exemplary embodiments do not represent all the implementations consistent with the present disclosure. Rather, they are merely examples in accordance with some aspects of the present disclosure as detailed in the appended claims.

[0028] It should be noted that all the actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0029] Figure 1 A structural schematic diagram of a computer system according to an exemplary embodiment of the present disclosure is shown, which includes a terminal 120 and a server 140.

[0030] The terminal 120 and the server 140 are connected to each other through a wired or wireless network.

[0031] The terminal 120 can include at least one of a smartphone, a notebook computer, a desktop computer, a tablet computer, a smart speaker, and a smart robot.

[0032] The terminal 120 includes a display; the display can be used to display the detection result of the associated model.

[0033] The terminal 120 comprises a first memory and a first processor. The first memory stores a first program, and the first program is invoked by the first processor to implement the method for determining model performance provided by the present disclosure. The first memory can include, but is not limited to, the following: Random Access Memory (RAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electric Erasable Programmable Read-Only Memory (EEPROM).

[0034] The first processor can be composed of one or more integrated circuit chips. Alternatively, the first processor can be a general-purpose processor, such as a Central Processing Unit (CPU) or a Network Processor (NP). For example, the trained association model in the terminal can be trained by the terminal; or trained by the server and obtained by the terminal from the server.

[0035] The server 140 comprises a second memory and a second processor. The second memory stores a second program, and the second program is invoked by the second processor to implement the method for determining model performance provided by the present disclosure. Alternatively, the second memory can include, but is not limited to, the following: RAM, ROM, PROM, EPROM, EEPROM. Alternatively, the second processor can be a general-purpose processor, such as a CPU or an NP.

[0036] The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the present disclosure.

[0037] For example, please refer to Figure 2 , Figure 2is a schematic diagram of a driving scene of a target vehicle provided by an example embodiment of the present disclosure. On two two-way roads separated by a lawn, the target vehicle 10 (ego vehicle) is driving on the right road, and the target vehicle 10 only needs to identify vehicles on the right road. The identification of vehicles on the left road is not practically meaningful for the driving of the target vehicle. It can reflect the performance of the vehicle detection model to some extent, but in fact, the accuracy of the identification of vehicles on the right road is more meaningful to the driving of the target vehicle 10. That is, assuming that there are 100 vehicles on the left side and 100 vehicles on the right side, one vehicle detection model has 10 missed detections on the left side and 5 false detections on the right side, and another vehicle detection model has 20 missed detections on the left side and 0 false detections on the right side. Therefore, from the overall point of view, the second vehicle detection model has a higher missed detection rate, but in fact, the second vehicle detection model has better actual effect. In the actual scene, because of the lane direction (vehicle recognition is affected by the vehicle head and tail) and the distance (the left vehicle is smaller, and the pixel resolution is low), it is easy to have such a result.

[0038] Based on the above reasons, the present disclosure provides a model performance determination method. Figure 3 is a flowchart of a model performance determination method according to an example embodiment, which is executed by a computer device, for example, by a terminal or a server in the computer system shown in Figure 1 , the model performance determination method shown in Figure 3 includes the following steps:

[0039] In step S101, the position of the target vehicle is obtained.

[0040] For example, as shown in Figure 2 , when the target vehicle is driving on the right side of the two-way road, the real-time image of the scene in the driving process of the target vehicle in the visual angle can be obtained by a camera and / or a laser radar device, as a to-be-detected image. According to the to-be-detected image combined with some vehicle positioning technology, the position of the target vehicle is located; the target vehicle is the vehicle in which the target vehicle is located, or it can be called ego vehicle.

[0041] In step S102, a plurality of attention regions in the to-be-detected image are divided according to the position of the target vehicle.

[0042] Exemplarily, the attention region is a region in the to-be-detected image containing different entities, and the entity can be a vehicle / pedestrian, a traffic light, a road sign, an obstacle, or a traffic sign in the to-be-detected image, and the like. For example, the attention region can include a traffic light attention region, a road sign attention region, a vehicle / pedestrian detection attention region, an obstacle attention region, a traffic sign attention region, and / or a trajectory prediction attention region. The traffic light attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the traffic light in the to-be-detected image, or can be referred to as a region where the traffic light in the to-be-detected image is located. The road sign attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the road sign in the to-be-detected image, or can be referred to as a region where the road sign in the to-be-detected image is located. The vehicle / pedestrian attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the vehicle / pedestrian in the to-be-detected image, or can be referred to as a region where the vehicle / pedestrian in the to-be-detected image is located. Similarly, the obstacle attention region and the traffic sign attention region correspond to the region where the obstacle in the to-be-detected image is located and the region where the traffic sign in the to-be-detected image is located, respectively. It should be noted that the trajectory prediction attention region corresponds to an associated model that needs to pay attention to a region for predicting the trajectory of a target vehicle according to the to-be-detected image. The region for predicting the trajectory of the target vehicle includes the road sign attention region, the vehicle / pedestrian detection attention region, the obstacle attention region, and the traffic sign attention region.

[0043] In step S103, the plurality of associated models are used to perform target detection on the corresponding adjusted attention regions in the to-be-detected image, respectively, to obtain a plurality of first detection results.

[0044] The adjustment of the attention region aims to obtain a more accurate attention region, and thus the adjusted attention region can obtain a more accurate first detection result.

[0045] In this embodiment, the corresponding associated model performs target detection on the adjusted attention region, and thus obtains a detection result, which is recorded as a first detection result. The first detection result includes the accuracy and recall rate of the associated model, such as the accuracy and recall rate of the road sign detection model for identifying the road sign, the accuracy and recall rate of the traffic light detection model for identifying the traffic light, and the like.

[0046] In step S104, the performance of the plurality of associated models is determined according to the plurality of first detection results, respectively.

[0047] According to the first detection result, the performance of the plurality of correlation models is determined respectively. For example, the performance of the road sign detection model is determined according to the road sign detection result; the performance of the traffic light detection model is determined according to the traffic light detection result; the performance of the road sign detection model is determined according to the road sign detection result; the performance of the vehicle / pedestrian detection model is determined according to the vehicle / pedestrian detection result; the performance of the obstacle detection model is determined according to the obstacle detection result; the performance of the traffic sign detection model is determined according to the traffic sign detection result; and the performance of the trajectory prediction model is determined according to the trajectory prediction result.

[0048] The present disclosure shows another method for determining the performance of a model according to an exemplary embodiment. Please refer to Figure 4 , Figure 4 is a flowchart of a method for determining the performance of a model according to an exemplary embodiment, which is performed by a computer device, for example, by a terminal or a server in the computer system shown in Figure 1 , the method for determining the performance of a model shown in Figure 4 is consistent with the content of Figure 3 , and the undiscussed parts can refer to the content of Figure 3 , Figure 4 The method for determining the performance of a model shown in

[0049] In step S201, the position of the target vehicle is obtained.

[0050] For example, as shown in Figure 2 , when the target vehicle is driving on the right side of a two-way road, the real-time image of the scene in the target vehicle's visual field during driving can be obtained by a camera and / or a laser radar, etc., as a to-be-detected image. Then, a key frame is extracted from the to-be-detected image, the key frame is an image frame in multiple image frames that can represent the scene in the target vehicle's visual field; a feature point of the key frame is extracted, and the position of the target vehicle is located according to the feature point; for example, the position of the target vehicle in the road in the to-be-detected image can be determined according to the lane line and the shooting position (the position of taking the to-be-detected image), and after detecting the relationship between the target vehicle and the lane line, the position of the target vehicle on the right side of the road can be determined.

[0051] In step S202, a plurality of attention regions in the to-be-detected image are divided according to the position of the target vehicle.

[0052] For example, the attention region is a region in the to-be-detected image containing different entities, and the entity can be a vehicle / pedestrian, a traffic light, a road sign, an obstacle, or a traffic sign in the to-be-detected image, etc. For example, the attention region can include a traffic light attention region, a road sign attention region, a vehicle / pedestrian detection attention region, an obstacle attention region, a traffic sign attention region, and / or a trajectory prediction attention region. The traffic light attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the traffic light in the to-be-detected image, or can be referred to as a region where the traffic light in the to-be-detected image is located. The road sign attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the road sign in the to-be-detected image, or can be referred to as a region where the road sign in the to-be-detected image is located. The vehicle / pedestrian attention region is a region that needs to be paid attention to by the corresponding associated model for detecting the vehicle / pedestrian in the to-be-detected image, or can be referred to as a region where the vehicle / pedestrian in the to-be-detected image is located. Similarly, the obstacle attention region and the traffic sign attention region correspond to the region where the obstacle in the to-be-detected image is located and the region where the traffic sign in the to-be-detected image is located, respectively. It should be noted that the trajectory prediction attention region corresponds to an associated model for predicting a region that needs to be paid attention to by the target vehicle for predicting the trajectory of the target vehicle in the to-be-detected image. The region that needs to be paid attention to by the target vehicle for predicting the trajectory of the target vehicle in the to-be-detected image includes the road sign attention region, the vehicle / pedestrian detection attention region, the obstacle attention region, and the traffic sign attention region.

[0053] In the foregoing step, the position of the target vehicle has been obtained, and therefore each attention region can be divided according to the position of the target vehicle. For example, for the right-side driving vehicle, the task is to drive forward on the right side. For the vehicle detection model, the region where the vehicle on the right side is located is the attention region. The position of the target vehicle relative to the lane line has been located, and for the traffic light detection model, the traffic light above the right lane is the attention region. It should be noted that the attention region divided in combination with the position of the target vehicle is a coarse-grained division, and needs to be adjusted in the subsequent step to make the boundary of the attention region more accurate.

[0054] In step S203, the attention region is adjusted by the associated model corresponding to the attention region to obtain a plurality of adjusted attention regions.

[0055] It should be noted that each attention region corresponds to an associated model. For example, the traffic light attention region corresponds to a traffic light detection model, the road sign attention region corresponds to a road sign detection model, the vehicle / pedestrian detection attention region corresponds to a vehicle / pedestrian detection model, the obstacle attention region corresponds to an obstacle detection model, the traffic sign attention region corresponds to a traffic sign detection model, and the trajectory prediction attention region corresponds to a trajectory prediction model.

[0056] The step of adjusting the attention region through the corresponding associated model of the attention region comprises: respectively identifying the to-be-detected image through the corresponding associated model of the plurality of attention regions to obtain a plurality of second detection results, wherein the second detection result comprises position information of the plurality of attention regions of the to-be-detected image, and then adjusting the plurality of attention regions according to the plurality of second detection results.

[0057] For example, a traffic light attention region in the to-be-detected image is identified through a traffic light detection model, denoted as a second traffic light attention region, and then the traffic light attention region is adjusted according to the identified second traffic light attention region; for another example, a road sign attention region in the to-be-detected image can be identified through a road sign detection model, denoted as a second road sign attention region, and then the road sign attention region is adjusted according to the identified second road sign attention region; and the adjustment mode of the vehicle / pedestrian detection attention region, the obstacle attention region, the traffic sign attention region and the trajectory prediction attention region is the same, which will not be described here.

[0058] In step S204, target detection is performed on the corresponding adjusted attention region in the to-be-detected image through the plurality of associated models to obtain a plurality of first detection results.

[0059] In the embodiment, different from step S203, step S203 is that the corresponding associated model detects the entire to-be-detected image, while the embodiment is that the corresponding associated model only performs target detection on the adjusted attention region to obtain a detection result, denoted as a first detection result. The first detection result comprises the accuracy and recall rate of the associated model, such as the accuracy and recall rate of the road sign detection model to identify the road sign, the accuracy and recall rate of the traffic light detection model to identify the traffic light, and the like.

[0060] In step S205, the performance of the plurality of associated models is determined according to the plurality of first detection results.

[0061] The performance of the plurality of associated models is determined according to the first detection result, for example, the performance of the road sign detection model is determined according to the road sign detection result; the performance of the traffic light detection model is determined according to the traffic light detection result; the performance of the road sign detection model is determined according to the road sign detection result; the performance of the vehicle / pedestrian detection model is determined according to the vehicle / pedestrian detection result; the performance of the obstacle detection model is determined according to the obstacle detection result; the performance of the traffic sign detection model is determined according to the traffic sign detection result; and the performance of the trajectory prediction model is determined according to the trajectory prediction result.

[0062] In the related art, the performance of the model is measured according to the detection result of the entire image to be detected identified by the model. In fact, the detection of unnecessary areas in the image to be detected is unnecessary. For example, the right driving vehicle, as described above, the identification of the vehicle on the left side is unnecessary. In the present disclosure, the attention area is screened according to the position of the target vehicle and the task scene of the target vehicle, such as the right driving vehicle. The task of the vehicle is to drive forward on the right side of the road. For the vehicle detection model, the vehicle on the right side is the attention area of the vehicle detection model. Then the vehicle detection model identifies the attention area to obtain the identification result. The performance of the vehicle detection model is determined according to the identification result.

[0063] For example, refer to Figure 5 , Figure 5 is a flow chart of another method for determining the performance of a model according to an example embodiment. Figure 5 In the present disclosure, first, the position of the ego vehicle is determined, that is, the position of the target vehicle is obtained. Then, the area is decomposed, that is, the attention area is divided according to the position of the target vehicle. For example, the attention area can be divided into traffic light attention area, road sign attention area, vehicle / pedestrian detection attention area, obstacle attention area, traffic sign attention area and trajectory prediction attention area. The associated models corresponding to these attention areas are traffic light attention model, road sign attention model, vehicle / pedestrian detection attention model, obstacle attention model, traffic sign attention model and trajectory prediction model. The associated models identify the corresponding attention areas to obtain traffic light detection result, road sign detection result, vehicle / pedestrian detection result, obstacle detection result, traffic sign detection result and trajectory prediction result. Then, the performance of the corresponding model is evaluated according to these detection results.

[0064] In summary, the method for determining the performance of the model provided by the present disclosure includes obtaining the position of the target vehicle, dividing a plurality of attention areas in the image to be detected according to the position of the target vehicle, the attention area being a region containing different entities in the image to be detected, using a plurality of associated models to perform target detection on the corresponding adjusted attention areas in the image to be detected to obtain a plurality of first detection results, one attention area corresponding to one associated model, and determining the performance of the plurality of associated models according to the first detection results; screening the attention area according to the position of the target vehicle and the task scene of the target vehicle, and then identifying the screened attention area using the corresponding associated model to obtain the identification result. The identification result fused with the position and task scene of the target vehicle can more accurately reflect the real performance of the associated model.

[0065] Figure 6 is a block diagram of a model performance determination device according to an example embodiment. Refer to Figure 6 The device 20 includes an acquisition module 201, a processing module 202 and a detection module 204.

[0066] The acquisition module 201 is configured to acquire a location of a target vehicle;

[0067] The processing module 202 is configured to divide a plurality of attention regions in the to-be-detected image according to the location of the target vehicle; the plurality of attention regions are regions containing different entities in the to-be-detected image;

[0068] The detection module 204 is configured to perform target detection on corresponding adjusted attention regions in the to-be-detected image respectively by using a plurality of correlation models to obtain a plurality of first detection results; one attention region corresponds to one correlation model.

[0069] The processing module 202 is further configured to determine the performance of the plurality of correlation models respectively according to the plurality of first detection results.

[0070] Optionally, the processing module 202 is further configured to adjust the attention regions by using the corresponding correlation models of the attention regions to obtain a plurality of adjusted attention regions.

[0071] Optionally, the acquisition module is further configured to acquire the to-be-detected image according to a camera and / or a laser radar; and locate the position of the target vehicle according to the to-be-detected image.

[0072] Optionally, the plurality of attention regions include a traffic light attention region, a road sign attention region, a vehicle / pedestrian detection attention region, an obstacle attention region, a traffic sign attention region, and / or a trajectory prediction attention region.

[0073] The processing module is further configured to locate the positions of the traffic light attention region, the road sign attention region, the vehicle / pedestrian detection attention region, the obstacle attention region, the traffic sign attention region, and / or the trajectory prediction attention region according to the location of the target vehicle.

[0074] Optionally, the processing module is further configured to identify the to-be-detected image by using the corresponding correlation models of the plurality of attention regions respectively to obtain a plurality of second detection results; the plurality of second detection results include position information of the plurality of attention regions in the to-be-detected image.

[0075] Adjust the plurality of attention regions according to the plurality of second detection results.

[0076] Optionally, the processing module is further configured to determine the performance of the association model according to a corresponding first detection result, the corresponding first detection result being obtained by the association model performing target detection on a corresponding attention region in the to-be-detected image, and the corresponding first detection result including an accuracy rate and a recall rate.

[0077] With regard to the apparatus in the above-described embodiments, specific manners in which various modules perform operations have been described in detail in embodiments of the method, and thus will not be described in detail here.

[0078] The present disclosure also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the method for determining model performance provided by the present disclosure.

[0079] Figure 7 is a block diagram of an apparatus 800 for determining model performance according to an exemplary embodiment. For example, the apparatus 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0080] Referring to Figure 7 , the apparatus 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0081] The processing component 802 usually controls overall operations of the apparatus 800, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described method for determining model performance. Further, the processing component 802 can include one or more modules so as to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0082] The memory 804 is configured to store various types of data to support operations of the device 800. Examples of such data include instructions for any application or methods operating on the device 800, contact data, phonebook data, messages, pictures, videos, and so on. The memory 804 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0083] The power supply component 806 supplies electrical power for various components of the device 800. The power supply component 806 can include a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing electrical power for the device 800.

[0084] The multimedia component 808 includes a screen providing an output interface between the device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touch or swiping action, but also detect duration and pressure associated with the touch or swiping action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 800 is in an operation mode such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0085] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive an external audio signal when the device 800 is in an operation mode such as a calling mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0086] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules such as a keyboard, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0087] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and keypad of the device 800, a change in position of the device 800 or a component of the device 800, presence or absence of user contact with the device 800, orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor component 814 can include proximity sensor(s) configured to detect presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0088] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and another device. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0089] In an exemplary embodiment, the device 800 can be implemented using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, other electronic units, or a combination thereof, to perform the method of determining model performance described above.

[0090] In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the device 800 to perform the method of determining model performance described above. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0091] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

[0092] It will be understood that the present disclosure is not limited to the precise structures hereinbefore described and illustrated in the drawings, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method for determining model performance, characterized in that, include: Obtain the location of the target vehicle; The image to be detected is divided into multiple regions of interest based on the location of the target vehicle; the multiple regions of interest are regions in the image to be detected that contain different entities. Multiple association models are used to perform target detection on the corresponding and adjusted regions of interest in the image to be detected, resulting in multiple first detection results; each region of interest corresponds to one association model. The performance of the multiple association models is determined based on the multiple first detection results; The method for adjusting the region of interest includes: adjusting the region of interest through the association model corresponding to the region of interest to obtain multiple adjusted regions of interest; The step of adjusting the region of interest using the association model corresponding to the region of interest to obtain multiple adjusted regions of interest includes: The image to be detected is identified by the corresponding association models of multiple regions of interest, and multiple second detection results are obtained; the multiple second detection results include the location information of multiple regions of interest in the image to be detected. The multiple regions of interest are adjusted based on the multiple second detection results.

2. The method according to claim 1, characterized in that, The steps for obtaining the location of the target vehicle include: The image to be detected is acquired using a camera and / or lidar; The location of the target vehicle is determined based on the image to be detected.

3. The method according to claim 2, characterized in that, The multiple areas of interest include traffic light areas of interest, road sign areas of interest, vehicle / pedestrian detection areas of interest, obstacle areas of interest, traffic sign areas of interest, and / or trajectory prediction areas of interest; The step of dividing the image to be detected into multiple regions of interest based on the location of the target vehicle includes: The location of the traffic light attention area, the road sign attention area, the vehicle / pedestrian detection attention area, the obstacle attention area, the traffic sign attention area, and / or the trajectory prediction attention area is determined based on the location of the target vehicle.

4. The method according to claim 1, characterized in that, The step of determining the performance of the multiple association models based on the multiple first detection results includes: The performance of the association model is determined based on the corresponding first detection result; the corresponding first detection result is obtained by the association model performing target detection on the corresponding region of interest in the image to be detected; the corresponding first detection result includes accuracy and recall.

5. A device for determining model performance, characterized in that, include: The acquisition module is configured to acquire the location of the target vehicle. The processing module is configured to divide the image to be detected into multiple regions of interest based on the location of the target vehicle; the multiple regions of interest are regions in the image to be detected that contain different entities. The detection module is configured to perform target detection on the corresponding adjusted regions of interest in the image to be detected using multiple association models, and obtain multiple first detection results; one region of interest corresponds to one association model; The processing module is further configured to determine the performance of the plurality of association models based on the plurality of first detection results; The method for adjusting the region of interest includes: adjusting the region of interest through the association model corresponding to the region of interest to obtain multiple adjusted regions of interest; The step of adjusting the region of interest using the association model corresponding to the region of interest to obtain multiple adjusted regions of interest includes: The image to be detected is identified by the corresponding association models of multiple regions of interest, and multiple second detection results are obtained; the multiple second detection results include the location information of multiple regions of interest in the image to be detected. The multiple regions of interest are adjusted based on the multiple second detection results.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 4.

Images