Image sample acquisition method, device, electronic device and storage medium

Abstract

The present application discloses an image sample acquisition method, device, electronic device and storage medium, and relates to the field of computer technology, and in particular to artificial intelligence fields such as autonomous driving, vehicle networking, smart cockpit, computer vision, and deep learning. The implementation scheme is: obtaining an image from a first vehicle terminal and a first detection result of the image; using a first detection model to detect the image to obtain a second detection result of the image; and determining the sample set to which the image belongs based on the comparison result between the first detection result and the second detection result. The method reduces the cost of image sample acquisition by determining the sample set to which the image belongs based on the comparison result between the detection result from the vehicle terminal and the detection result of the first detection model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to artificial intelligence fields such as autonomous driving, vehicle networking, smart cockpits, computer vision, deep learning, etc., and specifically to an image sample acquisition method, device, electronic device and storage medium. Background Art

[0002] In visual recognition, AR (Augmented Reality) navigation, etc., the collection of image samples has always been a difficult task, and the cost of collecting image samples simply through manual labeling is also relatively high. Summary of the invention

[0003] The present application provides an image sample acquisition method, device, electronic device and storage medium.

[0004] According to one aspect of the present application, a method for collecting image samples is provided, comprising:

[0005] Acquire an image from a first vehicle terminal and a first detection result of the image;

[0006] Using the first detection model to detect the image to obtain a second detection result of the image;

[0007] The sample set to which the image belongs is determined according to a comparison result between the first detection result and the second detection result.

[0008] According to another aspect of the present application, there is provided an image sample acquisition device, comprising:

[0009] A first acquisition module, used to acquire an image from a first vehicle terminal and a first detection result of the image;

[0010] A second acquisition module, used to detect the image using the first detection model to obtain a second detection result of the image;

[0011] A determination module is used to determine the sample set to which the image belongs based on a comparison result between the first detection result and the second detection result.

[0012] According to another aspect of the present application, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0016] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0017] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0020] Figure 1 A schematic diagram of a flow chart of an image sample acquisition method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a flow chart of an image sample acquisition method provided in another embodiment of the present application;

[0022] Figure 3 A schematic diagram of a flow chart of an image sample acquisition method provided in another embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of an image sample acquisition device provided in one embodiment of the present application;

[0024] Figure 5 It is a block diagram of an electronic device used to implement the image sample acquisition method of the embodiment of the present application. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] The following describes the image sample acquisition method, device, electronic device and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0027] Artificial intelligence is a discipline that studies the use of computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technical fields and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies include computer vision technology, speech recognition technology, natural language processing technology, as well as deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0028] The concept of Internet of Vehicles originates from the Internet of Things, namely the Internet of Vehicles. It takes moving vehicles as information perception objects and uses the new generation of information and communication technology to realize network connection between vehicles and X (i.e. vehicles, people, roads, and service platforms), improve the overall intelligent driving level of vehicles, and provide users with safe, comfortable, intelligent, and efficient driving experience and transportation services. At the same time, it improves traffic operation efficiency and enhances the intelligence level of social transportation services.

[0029] A smart cockpit refers to the transformation of the vehicle's interior seating space to make the driving and riding experience more comfortable and intelligent.

[0030] Computer vision is a science that studies how to make machines "see". It refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing so that the computer processes the images into images that are more suitable for human eye observation or transmission to instruments for detection.

[0031] Deep learning is a new research direction in the field of machine learning. Deep learning is the study of the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds.

[0032] Figure 1 A schematic diagram of a flow chart of an image sample acquisition method provided in an embodiment of the present application.

[0033] The image sample collection method of the embodiment of the present application can be executed by the server to compare the detection result of the vehicle-side model with the detection result of the first detection model of the server, and determine the sample set to which the image belongs based on the comparison result, thereby reducing the sample collection cost.

[0034] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0035] like Figure 1 As shown, the image sample acquisition method includes:

[0036] Step 101, obtaining an image from a first vehicle terminal and a first detection result of the image.

[0037] In the present application, a detection model may be provided in the first vehicle terminal, and the detection model may be used to detect the image to obtain the first detection result, or other detection methods may be used to detect the image to obtain the first detection result.

[0038] Taking the AR navigation scenario as an example, the detection model of the first vehicle terminal can be a model for detecting whether there is a vehicle in front in the image, or a model for detecting whether the image contains pedestrians, or a model for detecting whether the image contains non-motor vehicles, etc.

[0039] In the present application, a first vehicle terminal is installed on the vehicle, and images can be collected through a camera on the vehicle. The first vehicle terminal can use a detection model to detect the image to obtain a first detection result, and then the first vehicle terminal can send the image and the corresponding detection result to the server.

[0040] For example, after turning on the configuration item of road test data collection on the vehicle, the vehicle will automatically collect image data and the image detection model on the vehicle will detect the image to obtain the detection result during the road test. After the road test, if the network currently used by the vehicle is a wireless network, the user can be prompted to upload the image and the corresponding detection result, or can be directly uploaded. If the image and the corresponding detection result are not uploaded, the image and the corresponding detection result will be prompted to upload the image and the corresponding detection result the next time the wireless connection is established.

[0041] In the present application, the first vehicle terminal may also detect each frame image in the captured video, obtain a first detection result for each frame image, and send the video and the first detection result of each frame image in the video to the server.

[0042] Step 102: Use the first detection model to detect the image to obtain a second detection result of the image.

[0043] In the present application, a first detection model is provided on the server, and the server can use the first detection model to detect the image to obtain a second detection result. If the first vehicle terminal uses the detection model to detect the image to obtain the first detection result, then the function of the first detection model is the same as that of the detection model used by the first vehicle terminal, for example, the first detection model and the detection model used by the first vehicle terminal are both models for detecting whether the image contains pedestrians.

[0044] After the server obtains the image from the first vehicle terminal and the first detection result of the image, the server may detect the image using the first detection model to obtain a second detection result of the image.

[0045] Step 103: Determine the sample set to which the image belongs based on the comparison result between the first detection result and the second detection result.

[0046] In the present application, the first detection result and the second detection result may be compared, and the sample set to which the image belongs may be determined according to the comparison result of the two, so as to classify the image into the corresponding sample set.

[0047] The sample set to which the image belongs may be one of a standard sample set, a falsely detected sample set, a misdetected sample set, a missed-detected sample set, and the like.

[0048] For example, if the first detection result is consistent with the second detection result, the first detection result can be considered correct, and the sample set to which it belongs can be determined to be the standard sample set. For another example, the first confidence of the first detection result of the image from the vehicle terminal and the second confidence of the second detection result output by the first detection model can also be obtained. If the first detection result is the same as the second detection result, and the first confidence and the second confidence are both greater than the preset threshold, the sample set to which the image belongs is determined to be the standard sample set.

[0049] In order to improve the accuracy of image sample collection, in the present application, the server may also compare the first detection result of each image within a continuous period of time with the second detection result. If the first detection result of each image is the same as the second detection result, then one frame of image may be selected from each frame of image and added to the standard sample set.

[0050] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of this application are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0051] In the embodiment of the present application, an image and a first detection result of the image are obtained from a first vehicle terminal; a first detection model is used to detect the image to obtain a second detection result of the image; and the sample set to which the image belongs is determined based on a comparison result between the first detection result and the second detection result. Thus, by determining the sample set to which the image belongs based on a comparison result between the detection result from the vehicle terminal and the detection result of the first detection model, the cost of image sample acquisition is reduced.

[0052] Figure 2 A schematic diagram of a flow chart of an image sample acquisition method provided in another embodiment of the present application.

[0053] like Figure 2 As shown, the image sample acquisition method includes:

[0054] Step 201, obtaining an image from a first vehicle terminal and a first detection result of the image.

[0055] In the present application, a detection model is provided in the first vehicle machine end, which can be referred to as a second detection model for the sake of distinction. The second detection model can be obtained by compressing the first detection model.

[0056] For example, a large amount of data can be used to train a first detection model with relatively high accuracy. The first detection model can be compressed, such as by pruning, distillation, quantization, etc., to obtain a second detection model that is easy to run on the vehicle side. In the process of compressing the first detection model, the accuracy of the model will decrease. Therefore, the evaluation index value of the first detection model will be higher than that of the second detection model, that is, the accuracy of the detection result of the first detection model will be higher than that of the second detection model. Therefore, the second detection model is obtained by compressing the first detection model, which enriches the way to obtain the vehicle side model.

[0057] It should be noted that the first vehicle terminal may have multiple image detection models. The first vehicle terminal may use the multiple detection models to detect the image, obtain corresponding detection results, and send the image and the detection results of each model to the server.

[0058] Step 202: Use the first detection model to detect the image to obtain a second detection result of the image.

[0059] In the present application, step 202 is similar to that described in the above embodiment, so it will not be repeated here.

[0060] Since the second detection model is obtained by compressing the first detection model, the accuracy of the detection result of the first detection model is higher than the accuracy of the detection result of the second detection model. Therefore, the second detection result obtained by the first detection model can be used as a reference.

[0061] Step 203, obtaining a first confidence level of a first detection result from the first vehicle terminal and a second confidence level of a second detection result output by the first detection model.

[0062] In the present application, when the second detection model outputs the first detection result, it can also output the confidence of the first detection result, and the first vehicle terminal can also send the first confidence of the first detection result to the server, and the server can obtain the first confidence of the first detection result from the first vehicle terminal. The first confidence is used to characterize the credibility of the first detection result.

[0063] The first detection model of the server detects the image and, when outputting the second detection result, can also output the second confidence of the second detection result, so that the server can obtain the second confidence of the second detection result, where the second confidence is used to characterize the credibility of the second detection result.

[0064] Step 204 , when the first detection result is consistent with the second detection result and the first confidence level and the second confidence level are both greater than a preset threshold, determine that the sample set to which the image belongs is a standard sample set.

[0065] In the present application, the second detection result can be used as a reference to compare the first detection result with the second detection result. If the first detection result is consistent with the second detection result, and the first confidence level and the second confidence level are both greater than a preset threshold, the first detection result can be considered correct, and the sample set to which the image belongs can be determined to be the standard sample set. The image is added to the standard sample set, and the second detection result is used as an annotation for the image. No manual annotation is required, thereby reducing costs.

[0066] In the present application, images with correct detection results can be divided into a standard sample set, and the image samples in the standard sample set can increase the sample size to facilitate subsequent updating of the first detection model.

[0067] Step 205: When the first detection result is inconsistent with the second detection result, determine that the sample set to which the image belongs is a misdetected sample set.

[0068] In the present application, if the first detection result is inconsistent with the second detection result, it can be considered that the first detection result is incorrect, and the sample set to which the image belongs can be determined to be a wrong detection sample set. Among them, the image samples in the wrong detection sample set are images whose detection results of the second detection model are incorrect. Therefore, the wrong detection images can be divided into the wrong detection sample set, and the image samples in the wrong detection sample set can be used to test the updated second detection model and solve the wrong detection problem.

[0069] Step 206: When the first detection result is empty and the second detection result is not empty, determine that the sample set to which the image belongs is a missed detection sample set.

[0070] In this application, if the first detection result is empty, but the second detection result is not empty, it can be considered that the second detection model has not detected the image, that is, the image has been missed, and the sample set to which the image belongs can be determined to be a missed sample set. Among them, the image samples in the missed sample set are images missed by the second detection model.

[0071] Therefore, the missed detection images can be divided into a missed detection sample set, and the image samples in the missed detection sample set can be used to test the updated second detection model and the solution to the missed detection problem.

[0072] Step 207: when the first detection result is not empty and the second detection result is empty, determine that the sample set to which the image belongs is a false detection sample set.

[0073] In the present application, if the first detection result is not empty and the second detection result is empty, the image may be regarded as a falsely detected image, then the sample set to which the image belongs is determined to be a falsely detected sample set, and the image is added to the falsely detected sample set.

[0074] Therefore, the falsely detected images can be divided into a falsely detected sample set, and the image samples in the falsely detected sample set can be used to test the updated second detection model and the solution to the false detection problem.

[0075] In the present application, the second detection model on the first vehicle terminal is in a one-to-one correspondence with the first detection model on the server. For example, the first vehicle terminal has a model for detecting whether an image contains pedestrians and a model for detecting whether an image contains non-motor vehicles, and the server also has two models. If the first vehicle terminal sends the detection results of the two models to the server, the server can compare the detection result of whether the image detected by the first vehicle terminal contains pedestrians with the detection result of the same model on the server, and determine the sample set to which the image corresponding to the detection model on the first vehicle terminal belongs according to the comparison result, and compare the detection result of whether the image detected by the first vehicle terminal contains non-motor vehicles with the detection result of the same model on the server, and determine the sample set to which the image corresponding to the detection model on the first vehicle terminal belongs according to the comparison result.

[0076] In an embodiment of the present application, the sample set to which the image belongs can be determined based on whether the first detection result is consistent with the second detection result, and the image can be divided into corresponding sample sets, so as to collect samples of different types, which is convenient for subsequent optimization of the second detection model.

[0077] Figure 3A schematic diagram of a flow chart of an image sample acquisition method provided in another embodiment of the present application.

[0078] like Figure 3 As shown, the image sample acquisition method includes:

[0079] Step 301, obtaining an image from a first vehicle terminal and a first detection result of the image.

[0080] Step 302: Use the first detection model to detect the image to obtain a second detection result of the image.

[0081] In the present application, step 301 - step 302 are similar to the contents recorded in the above embodiment, so they will not be repeated here.

[0082] Step 303, obtaining the current state of the second vehicle terminal, wherein the second vehicle terminal has a first detection model.

[0083] In order to improve the accuracy of image acquisition, in the present application, other vehicle terminals can be used to detect the image, and based on the detection results of multiple vehicle terminals, the sample set to which the image belongs can be determined.

[0084] In the present application, the second vehicle terminal may report its status to the server at preset intervals. Alternatively, the server may send a status query request to the second vehicle terminal, and the second vehicle terminal returns the current status to the server, and the server may obtain the current status of the second vehicle terminal.

[0085] In actual applications, the vehicle terminal is usually idle during certain time periods. Based on this, the server can send a status query request to the second vehicle terminal during a preset time period, such as from 8 pm to 4 am the next day.

[0086] It should be noted that the second vehicle terminal may be one or more, and this application does not limit this.

[0087] Step 304: When the current state is an idle state, send the image to the second vehicle terminal.

[0088] In the present application, when the second vehicle terminal is in an idle state, the service end may send an image to the second vehicle terminal so that the second vehicle terminal detects the image.

[0089] In order to improve efficiency, the second vehicle terminal can be pre-registered with the server and the automatic detection function can be turned on. The server can send the image and the corresponding automatic detection script to the second vehicle terminal. The second vehicle terminal obtains the image and the automatic detection script and turns on the automatic detection.

[0090] Step 305, obtaining a third detection result of the image from the second vehicle terminal.

[0091] In the present application, the second vehicle terminal may be provided with the same detection model as the first vehicle terminal, that is, the second detection model. The second vehicle terminal may use the second detection model to detect the image to obtain the third detection result of the image, or may use other methods to detect the image to obtain the third detection result. The second vehicle terminal sends the third detection result to the server, so that the server can obtain the third detection result of the image from the second vehicle terminal.

[0092] For example, the server obtains the image uploaded by the vehicle terminal A and the corresponding first detection result. When the server determines that the vehicle terminals B, C, and D are in an idle state, the server sends the image to the vehicle terminals B, C, and D respectively, and obtains the detection results of the image returned by the vehicle terminals B, C, and D.

[0093] It should be noted that, in the present application, step 302 and step 303-step 305 can be executed in the above order, or step 303-step 305 can be executed first and then step 302, and step 302 and step 303 can also be executed at the same time, and the present application does not limit this.

[0094] Step 306: Determine the sample set to which the image belongs based on the comparison result of the first detection result, the second detection result, and the third detection result.

[0095] In the present application, the first detection result, the second detection result and the third detection result can be compared, and the sample set to which the image belongs can be determined according to the comparison result. For example, if the first detection result, the second detection result and the third detection result are all the same, it can be determined that the sample set to which the image belongs is the standard sample set.

[0096] Alternatively, the first vehicle terminal and the second vehicle terminal can both be provided with a second detection model, and the second detection model is obtained by compressing the first detection model. Then, the second detection result of the server can be used as a reference, and the first detection result and the third detection result can be compared with the second detection result respectively.

[0097] If at least one of the first detection result and the third detection result is consistent with the second detection result, it can be determined that the sample set to which the image belongs is the standard sample set; if the first detection result and the third detection result are different from the second detection result, it means that the second detection model on the vehicle side has incorrect detection results for the image, and it is determined that the sample set to which the image belongs is the misdetected sample set, and the image is added to the misdetected sample set.

[0098] If the first and third detection results are both empty, but the second detection result is not empty, it can be determined that the sample set to which the image belongs is a missed detection sample set. If the first and third detection results are both not empty, but the second detection result of the first detection model is empty, it can be determined that the sample set to which the image belongs is a false detection sample set.

[0099] Therefore, the image can be divided into corresponding sample sets according to the comparison between the first detection result, the second detection result and the third detection result.

[0100] In this application, since both the first vehicle terminal and the second vehicle terminal use the second detection model to detect the image, and the second detection model is obtained by compressing the first detection model, if the first detection result is inconsistent with the second detection result, and the third detection result is consistent with the second detection result, it means that the inconsistency between the first detection result and the second detection result is not a problem of the second detection model itself, but caused by the low hardware performance of the first vehicle terminal, then the service end can send a prompt message to the first vehicle terminal. The prompt message is used to prompt the first vehicle terminal to update the hardware.

[0101] For example, the second detection model is used to detect whether the image contains non-motor vehicles. If the detection result of the first vehicle terminal A is that it does not contain non-motor vehicles, and the detection results of the second vehicle terminals B, C, and D all contain non-motor vehicles, and the detection result of the server is that it contains non-motor vehicles, it means that the detection result of the first vehicle terminal A is incorrect, which may be caused by poor hardware performance of the first vehicle terminal A.

[0102] Therefore, when it is determined based on the detection results of multiple vehicle terminals and the detection results of the service end that the error in the detection result is caused by the poor hardware performance of the first vehicle terminal, hardware update prompt information is sent to the first vehicle terminal, thereby improving intelligence.

[0103] If the first detection result and the third detection result are different from the second detection result, it means that the detection result is incorrect due to the second detection model itself. Then the server can send a prompt message to the first vehicle terminal and the second vehicle terminal to prompt the vehicle terminal to update the second detection model.

[0104] Therefore, by obtaining the detection results under different hardware environments, comparing the detection results under different hardware environments with the detection results of the server, and determining the sample set to which the image belongs based on the comparison results, it is possible to avoid deviations in image sample collection due to a single hardware difference and improve the reliability of the results.

[0105] In the embodiment of the present application, the current state of the second vehicle terminal can also be obtained. When the current state is an idle state, an image is sent to the second vehicle terminal to obtain a third detection result of the image from the second vehicle terminal, and the sample set to which the image belongs is determined based on the comparison results of the first detection result, the second detection result, and the third detection result. Thus, by obtaining the detection results of multiple vehicle terminals, comparing the detection results of multiple vehicle terminals with the detection results of the server, and determining the sample set to which the image belongs based on the comparison results, the reliability of the results is improved.

[0106] In order to implement the above embodiment, the embodiment of the present application also proposes an image sample acquisition device. Figure 4 A schematic diagram of the structure of an image sample acquisition device provided in one embodiment of the present application.

[0107] like Figure 4 As shown, the image sample acquisition device 400 includes:

[0108] A first acquisition module 410 is used to acquire an image and a first detection result of the image from a first vehicle terminal;

[0109] A second acquisition module 420 is used to detect the image using the first detection model to obtain a second detection result of the image;

[0110] The determination module 430 is used to determine the sample set to which the image belongs according to the comparison result between the first detection result and the second detection result.

[0111] In a possible implementation of the embodiment of the present application, the device may further include:

[0112] A third acquisition module, used to acquire a first confidence level of a first detection result from the first vehicle terminal and a second confidence level of a second detection result output by the first detection model;

[0113] The determination module 430 is used to determine that the sample set to which the image belongs is a standard sample set when the first detection result is consistent with the second detection result and the first confidence level and the second confidence level are both greater than a preset threshold.

[0114] In a possible implementation of the embodiment of the present application, the first detection result is obtained by the first vehicle terminal detecting the image using a second detection model, and the second detection model is obtained by compressing the first detection model.

[0115] In a possible implementation of the embodiment of the present application, the determination module 430 is used to:

[0116] When the first detection result is inconsistent with the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

[0117] In a possible implementation of the embodiment of the present application, the determination module 430 is used to:

[0118] When the first detection result is empty and the second detection result is not empty, it is determined that the sample set to which the image belongs is a missed detection sample set.

[0119] In a possible implementation of the embodiment of the present application, the determination module 430 is used to:

[0120] When the first detection result is not empty and the second detection result is empty, it is determined that the sample set to which the image belongs is a false detection sample set.

[0121] In a possible implementation of the embodiment of the present application, the device may further include:

[0122] The fourth acquisition module is used to obtain the current state of the second vehicle terminal;

[0123] A sending module, used for sending an image to a second vehicle terminal when the vehicle terminal is in an idle state;

[0124] The fourth acquisition module is further used to acquire a third detection result of the image from the second vehicle terminal;

[0125] The determination module 430 is used to determine the sample set to which the image belongs according to the comparison result of the first detection result, the second detection result and the third detection result.

[0126] In a possible implementation of the embodiment of the present application, the first detection result and the third detection result are both obtained by detecting the image using the second detection model, and the second detection model is obtained by compressing the first detection model; the determination module 430 is used to:

[0127] When at least one of the first detection result and the third detection result is consistent with the second detection result, determining that the sample set to which the image belongs is the standard sample set;

[0128] When the first detection result and the third detection result are all different from the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

[0129] In a possible implementation manner of the embodiment of the present application, the sending module is further used to:

[0130] When the first detection result is inconsistent with the second detection result and the third detection result is consistent with the second detection result, a prompt message is sent to the first vehicle machine end, wherein the prompt message is used to prompt the first vehicle machine end to perform a hardware update.

[0131] It should be noted that the explanation of the above-mentioned embodiment of the image sample acquisition method is also applicable to the image sample acquisition device of this embodiment, so it will not be repeated here.

[0132] In the embodiment of the present application, an image and a first detection result of the image are obtained from a first vehicle terminal; a first detection model is used to detect the image to obtain a second detection result of the image; and the sample set to which the image belongs is determined based on a comparison result between the first detection result and the second detection result. Thus, by determining the sample set to which the image belongs based on a comparison result between the detection result from the vehicle terminal and the detection result of the first detection model, the cost of image sample acquisition is reduced.

[0133] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0134] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0135] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.

[0136] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the image sample acquisition method. For example, in some embodiments, the image sample acquisition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the image sample acquisition method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the image sample acquisition method in any other appropriate manner (eg, by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0143] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0144] According to an embodiment of the present application, the present application further provides a computer program product, which, when an instruction processor in the computer program product executes, executes the image sample acquisition method proposed in the above embodiment of the present application.

[0145] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0146] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for collecting image samples, comprising: Acquire an image from a first vehicle terminal and a first detection result of the image; Using the first detection model to detect the image to obtain a second detection result of the image; Determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result; Obtaining a first confidence level of the first detection result from the first vehicle terminal and a second confidence level of the second detection result output by the first detection model; The step of determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result includes: When the first detection result is consistent with the second detection result and the first confidence level and the second confidence level are both greater than a preset threshold, it is determined that the sample set to which the image belongs is a standard sample set.

2. The method of claim 1, wherein: The first detection result is obtained by the first vehicle machine end detecting the image using a second detection model, and the second detection model is obtained by compressing the first detection model.

3. The method of claim 2, wherein: The step of determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result includes: When the first detection result is inconsistent with the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

4. The method of claim 2, wherein: The step of determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result includes: When the first detection result is empty and the second detection result is not empty, it is determined that the sample set to which the image belongs is a missed detection sample set.

5. The method of claim 2, wherein: The step of determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result includes: When the first detection result is not empty and the second detection result is empty, it is determined that the sample set to which the image belongs is a false detection sample set.

6. The method of claim 1, wherein: After acquiring the image from the first vehicle terminal and the first detection result of the image, the method further includes: Obtain the current state of the second vehicle terminal; When the state is an idle state, sending the image to the second vehicle terminal; Acquire a third detection result of the image from the second vehicle terminal; The step of determining the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result includes: The sample set to which the image belongs is determined according to a comparison result of the first detection result, the second detection result, and the third detection result.

7. The method of claim 6, wherein: The first detection result and the third detection result are both obtained by detecting the image using a second detection model, and the second detection model is obtained by compressing the first detection model; The determining, according to the comparison result of the first detection result, the second detection result and the third detection result, the sample set to which the image belongs includes: When at least one of the first detection result and the third detection result is consistent with the second detection result, determining that the sample set to which the image belongs is a standard sample set; When the first detection result and the third detection result are all different from the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

8. The method of claim 7, further comprising: When the first detection result is inconsistent with the second detection result and the third detection result is consistent with the second detection result, a prompt message is sent to the first vehicle machine end, wherein the prompt message is used to prompt the first vehicle machine end to perform a hardware update.

9. An image sample acquisition device, comprising: A first acquisition module, used to acquire an image from a first vehicle terminal and a first detection result of the image; A second acquisition module, used to detect the image using the first detection model to obtain a second detection result of the image; a determination module, configured to determine the sample set to which the image belongs according to a comparison result between the first detection result and the second detection result; a third acquisition module, configured to acquire a first confidence level of the first detection result from the first vehicle terminal and a second confidence level of the second detection result output by the first detection model; The determination module is used to determine that the sample set to which the image belongs is a standard sample set when the first detection result is consistent with the second detection result and the first confidence level and the second confidence level are both greater than a preset threshold.

10. The device of claim 9, wherein: The first detection result is obtained by the first vehicle machine end detecting the image using a second detection model, and the second detection model is obtained by compressing the first detection model.

11. The device of claim 10, wherein: The determining module is used to: When the first detection result is inconsistent with the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

12. The device of claim 10, wherein: The determining module is used to: When the first detection result is empty and the second detection result is not empty, it is determined that the sample set to which the image belongs is a missed detection sample set.

13. The device of claim 10, wherein: The determining module is used to: When the first detection result is not empty and the second detection result is empty, it is determined that the sample set to which the image belongs is a false detection sample set.

14. The apparatus of claim 9, further comprising: The fourth acquisition module is used to obtain the current state of the second vehicle terminal; A sending module, used for sending the image to the second vehicle terminal when the state is an idle state; The fourth acquisition module is further used to acquire a third detection result of the image from the second vehicle terminal; The determination module is used to determine the sample set to which the image belongs according to a comparison result of the first detection result, the second detection result and the third detection result.

15. The device of claim 14, wherein: The first detection result and the third detection result are both obtained by detecting the image using a second detection model, and the second detection model is obtained by compressing the first detection model; The determining module is used to: When at least one of the first detection result and the third detection result is consistent with the second detection result, determining that the sample set to which the image belongs is a standard sample set; When the first detection result and the third detection result are all different from the second detection result, it is determined that the sample set to which the image belongs is a misdetected sample set.

16. The device of claim 15, wherein: The sending module is further used for: When the first detection result is inconsistent with the second detection result and the third detection result is consistent with the second detection result, a prompt message is sent to the first vehicle machine end, wherein the prompt message is used to prompt the first vehicle machine end to perform a hardware update.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images