Image acquisition method, device, equipment, medium and product

Abstract

The invention discloses an image acquisition method, device and equipment, a medium and a product. The method comprises the steps of obtaining an image data set collected by a vehicle-mounted camera; performing anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and confidence of the anomaly detection type; obtaining a target image data set from the image data set according to the confidence; and determining label information of the target image data set according to the anomaly detection type, and storing target image data frames in the target image data set based on the label information so as to train an anomaly detection algorithm. According to the method, the target image data set can be obtained with a clear target, manual screening and classification are not needed, and the obtaining efficiency and effectiveness are improved; according to the method, the target image data frames are automatically classified and stored based on the label information, so that a targeted training sample is conveniently provided for training an anomaly detection algorithm, the blindness of anomaly detection algorithm training is avoided, and the iteration upgrading period of the anomaly detection algorithm is shortened.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image acquisition method, device, equipment, medium and product. Background Art

[0002] With the development of society, driving safety has gradually attracted people's attention. AI applications such as driver status monitoring DSM and driving condition monitoring ADAS have continued to emerge, bringing new opportunities for improving vehicle assisted driving functions.

[0003] In order to further improve the accuracy of AI algorithms and improve the vehicle assisted driving function, a large amount of high-quality real-life materials are needed to iteratively train each AI algorithm. Currently, common methods for collecting training materials include real-vehicle data collection, crowdsourcing platform data collection, and customer post-event data collection.

[0004] However, these methods have many shortcomings. The real-car data collection method is not only time-consuming and labor-intensive, but also requires manual screening and classification, which is inefficient; although the crowdsourcing platform data collection method can gather data from multiple parties, the data quality is uneven and it is difficult to meet the needs of precise training; the customer post-data collection method often faces problems such as historical material destruction and incomplete data, which seriously affects the validity and availability of the data. The existence of these problems has seriously restricted the optimization and upgrading of AI algorithms in the field of vehicle assisted driving. Summary of the invention

[0005] The present invention provides an image acquisition method, device, equipment, medium and product to solve the problems of unclear objectives, low efficiency and destruction of historical materials in the existing methods when collecting image training materials.

[0006] In a first aspect, an embodiment of the present invention provides an image acquisition method, which is applied to a vehicle-mounted video recording device, and the method includes:

[0007] Obtain image datasets collected by vehicle-mounted cameras;

[0008] Performing anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type;

[0009] Acquire a target image dataset from the image dataset according to the confidence level;

[0010] The label information of the target image data set is determined according to the anomaly detection type, and the target image data frames in the target image data set are stored based on the label information for training the anomaly detection algorithm.

[0011] In a second aspect, an embodiment of the present invention provides an image acquisition device, which is applied to a vehicle-mounted video recording device, and the device includes:

[0012] A first image data set acquisition module, used to acquire an image data set collected by a vehicle-mounted camera;

[0013] An anomaly detection module, used to perform anomaly detection on the image data set based on an anomaly detection algorithm, and obtain an anomaly detection type and a confidence level of the anomaly detection type;

[0014] a second image data set acquisition module, configured to acquire a target image data set from the image data set according to the confidence level;

[0015] The image data set storage module is used to determine the label information of the target image data set according to the anomaly detection type, and store the target image data frames in the target image data set based on the label information for training the anomaly detection algorithm.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, which serves as the vehicle-mounted video recording device, and the electronic device includes:

[0017] at least one processor;

[0018] and a memory communicatively coupled to the at least one processor;

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image acquisition method described in any embodiment of the present invention.

[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image acquisition method described in any embodiment of the present invention when executed.

[0021] In a fifth aspect, an embodiment of the present invention further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the image acquisition method according to any embodiment of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention is to obtain an image data set collected by a vehicle-mounted camera; perform anomaly detection on the image data set based on an anomaly detection algorithm to obtain anomaly detection types and confidence levels of the anomaly detection types; obtain a target image data set from the image data set based on the confidence levels; determine the label information of the target image data set based on the anomaly detection type, and store the target image data frames in the target image data set based on the label information for training the anomaly detection algorithm. The method obtains the target image data set based on the confidence levels after obtaining the anomaly detection type and the confidence levels of the anomaly detection type, and determines the label information of the target image data set based on the anomaly detection type, so that the target of obtaining the target image data set is clear, and manual screening and classification are not required, thereby improving the efficiency, cost, and effectiveness of obtaining the target image data set; by automatically classifying and storing the target image data frames based on the label information, targeted training samples are conveniently provided for training the anomaly detection algorithm, thereby avoiding the blindness of the training of the anomaly detection algorithm, shortening the iterative upgrade cycle of the anomaly detection algorithm, and improving the efficiency of improving the recognition effect of the anomaly detection algorithm.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flowchart of an image acquisition method provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the structure of an image acquisition device provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "target", "original", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should be noted that in order to further improve the existing AI algorithm's recognition accuracy of abnormal behaviors or abnormal situations that affect driving safety during vehicle operation, a large amount of high-quality and diverse real-time image materials are needed to iteratively train the AI ​​algorithm. However, the current common training material collection methods are inefficient, have uneven data quality, and have historical materials destroyed when collecting real-time image materials, which restricts the improvement of AI algorithm recognition accuracy and further affects the effective promotion of related business processes and user experience.

[0031] Based on this, an embodiment of the present invention provides an image acquisition method. Figure 1 A flowchart of an image acquisition method provided in an embodiment of the present invention. The embodiment of the present invention can be applicable to scenarios of acquiring image samples for training anomaly detection algorithms. The method can be executed by an image acquisition device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device serving as a vehicle-mounted video recording device, which is preferably a mobile terminal, a desktop computer, a laptop computer, a server, etc.

[0032] like Figure 1 As shown, the image acquisition method provided by the embodiment of the present invention may specifically include:

[0033] S101, obtaining an image data set collected by a vehicle-mounted camera.

[0034] It should be noted that each vehicle equipped with an on-board video recording device is also equipped with multiple on-board cameras that meet the resolution requirements. The on-board cameras can be installed at different locations such as the front, rear or side of the vehicle to obtain images of different perspectives of the vehicle's external road conditions. At the same time, the on-board cameras can also be installed inside the vehicle (such as above the instrument panel, above the center console, above the ceiling above the co-pilot, etc.) to obtain images of the driver and / or passengers. The images collected by each on-board camera are stored in the corresponding image data sets through different channels.

[0035] In this embodiment, an image data set consisting of image data frames inside and outside the vehicle collected in real time by each vehicle-mounted camera during driving is obtained. The image data frames can be in a format such as JPEG or PNG.

[0036] S102: Perform anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type.

[0037] Among them, the anomaly detection algorithm can be understood as an algorithm for identifying data points or samples that are significantly different from the expected pattern or normal behavior, such as point cloud matching algorithm, clustering algorithm, decision tree, support vector machine and deep learning network model, etc., which can be installed in advanced driver assistance system (ADAS) and driver state monitoring (DSM) system. The anomaly detection type can be understood as the category of abnormal behavior or abnormal situation identified by the anomaly detection algorithm. For example, the anomaly detection type can be abnormal driving behavior (such as smoking, fatigue, making phone calls, distraction, etc.), or dangerous vehicle conditions (such as too close distance, lane deviation, road failure, etc.), or abnormal road conditions. The confidence of the anomaly detection type can be understood as the probability of outputting the anomaly detection type accurately, which is a value between 0 and 1.

[0038] In this embodiment, an abnormality detection algorithm built into the vehicle-mounted video recording device is used to traverse the image data set to perform abnormality detection on each image data frame in the image data set. When the abnormality detection algorithm detects that a certain image data frame is abnormal, the corresponding abnormality detection type and the confidence corresponding to the abnormality detection type are output. At this time, the abnormality detection type output by the abnormality detection algorithm and the confidence corresponding to the abnormality detection type are obtained, such as 0.8 for making a phone call.

[0039] S103: Acquire a target image dataset from the image dataset according to the confidence level.

[0040] Among them, the target image dataset can be understood as an image dataset composed of image data frames that meet the collection requirements, which is used to provide effective image training samples for subsequent training of anomaly detection algorithms. The target image dataset can include image data frames and / or videos composed of continuous image data frames.

[0041] In this embodiment, a confidence threshold or confidence interval for acquiring a target image data set is preset, and when the preset confidence threshold or confidence interval is met, the target image data set is acquired from the image data set. Exemplarily, the manner of presetting the confidence threshold or confidence interval may be that when the anomaly detection algorithm determines that the confidence standard of the anomaly detection type is relatively high, the confidence threshold may be directly pre-set, such as 0.95. When the confidence is lower than 0.95, the output anomaly detection type is considered to have a certain degree of suspicion, and the target image data set is acquired at this time. When the anomaly detection algorithm determines that the confidence standard of the anomaly detection type is relatively low, the confidence interval may be pre-set to [0.7, 0.95]. When the confidence corresponding to the anomaly detection type is lower than 0.7, the output anomaly detection type is considered to be unreliable, and image data is not collected. When the confidence corresponding to the anomaly detection type is higher than 0.95, the output anomaly detection type is considered to be highly reliable, and image data is not collected. When the output anomaly detection type is in [0.7, 0.95], the output anomaly detection type is considered to have a certain degree of suspicion, and the target image data set is acquired at this time.

[0042] Continuing with the above description, a method for acquiring a target image data set from an image data set may be to determine an image data set in which an image data frame corresponding to an abnormality detection type is located, and to acquire one or more image data frames before and / or after the image data frame corresponding to the abnormality detection type in the image data set according to a preset frame interval. For example, if the preset frame interval is 50, if the number of acquired image data frames is 1, the image data frame corresponding to the abnormality detection type may be acquired, or an image data frame before or after the image data frame corresponding to the abnormality detection type and spaced 50 frames apart from the image data frame; if the number of acquired image data frames is 4, four image data frames after the image data frame corresponding to the abnormality detection type and spaced 50 frames, 100 frames, 150 frames, and 200 frames apart from the image data frame may be acquired to form a target image data set, or four image data frames before the image data frame corresponding to the abnormality detection type and spaced 50 frames and 100 frames apart from the image data frame may be acquired to form a target image data set. The method for obtaining the target image data set may also be to determine the image data set in which the image data frame corresponding to the anomaly detection type is located and the acquisition timestamp of the image data frame, and to obtain one or more image data frames after the acquisition timestamp in the image data set at a preset time interval. For example, the preset time interval is 0.2 seconds. If the number of image data frames obtained is 1, the image data frame corresponding to the anomaly detection type may be obtained, or an image data frame after the acquisition timestamp of the image data frame corresponding to the anomaly detection type and 0.2 seconds apart from the timestamp may be obtained. If the number of image data frames obtained is 3, three image data frames after the acquisition timestamp of the image data frame corresponding to the anomaly detection type and 0.2 seconds, 0.4 seconds, and 0.6 seconds apart from the timestamp may be obtained to form the target image data set. Alternatively, two image data frames before and after the acquisition timestamp of the image data frame corresponding to the anomaly detection type and 0.2 seconds apart from the timestamp may be obtained, as well as the image data frame corresponding to the anomaly detection type, to form the target image data set.

[0043] It should be noted that the preset frame interval and the preset time interval can be set according to empirical values, such as according to the duration of each abnormal detection type. For example, the duration of smoking is usually less than 2 seconds, so the frame interval or time interval can be set to a smaller time. The duration of a phone call is usually more than 10 seconds, so the frame interval or time interval can be set to a larger time.

[0044] Continuing with the above description, the number of image data frames to be acquired may be a preset fixed value, or different numbers may be configured for different confidence levels, or the confidence levels may be divided into levels and different numbers may be determined for each level.

[0045] In an optional embodiment, when a preset confidence threshold or confidence interval for acquiring a target image data set is met, if the confidence corresponding to the anomaly detection type is low (which can be determined by setting another confidence threshold or confidence interval), all image data frames within a continuous time after the image data frame corresponding to the anomaly detection type can also be acquired simultaneously to form a video, which together with the acquired image data frames at a specific frame interval or time interval constitute the target image data set.

[0046] S104: Determine label information of the target image data set according to the anomaly detection type, and store the target image data frames in the target image data set based on the label information for training anomaly detection algorithm.

[0047] The target image data frame may be understood as an image frame constituting a target image data set, and each target image data frame is a static image.

[0048] In this embodiment, corresponding label information may be pre-assigned to each abnormality detection type. For example, the corresponding label information assigned to the vehicle distance being too close may be the vehicle distance being too close, or may be TS vehicle distance being too close, or may be 1. After obtaining the abnormality detection type and the target image data set corresponding to the abnormality detection type, the label information of the target image data set is determined according to the abnormality detection type. For example, following the above example description, if the abnormality detection type is the vehicle distance being too close, the label information of the target image data set corresponding to the abnormality detection type may be the vehicle distance being too close, TS vehicle distance being too close, or 1. It can be understood that at this time, the label information corresponding to each target image data frame in the target image data set is the same as the target image data set.

[0049] Continuing with the above description, a corresponding storage path can also be determined in advance for each tag information, and the storage path can be a storage space in the corresponding vehicle video recording device, or a storage space in other devices or servers. When the target image data set is acquired, the target image data set with the same tag information can be stored in the same storage path in the vehicle video recording device, such as storing each target image data frame in the target image data set with the tag information TS vehicle distance is too close in the same folder named TS vehicle distance is too close; when the target image data set is acquired and the preset upload requirements are met (such as meeting the image clarity requirements and completing the priority sorting), the target image data frame can be automatically processed such as compressed, and then sent to other devices or servers for classified storage in sequence according to the storage path corresponding to the tag information.

[0050] It should be noted that the other device or server can simultaneously receive target image data frames sent by multiple vehicle-mounted video recording devices installed in different vehicles.

[0051] In this embodiment, when the algorithm updater (personnel) needs to train the anomaly detection algorithm to further improve the accuracy of the algorithm, the target image data frames stored in the storage path can be acquired in batches from the corresponding storage path of the above-mentioned server according to the requirements (such as the image samples whose label information is that the TS vehicle is too close). After acquiring the target image data frame, the algorithm updater (personnel) trains the anomaly detection algorithm according to the target image data frame and its corresponding label information. In an optional embodiment, the anomaly detection algorithm can be used to extract features and calculate feature similarity for each target image data frame, and remove target image data frames with low similarity to reduce the number of interfering image data frames and improve the efficiency, effectiveness and pertinence of training the anomaly detection algorithm.

[0052] Continuing with the above description, when the training of the anomaly detection algorithm is completed and an upgraded version of the anomaly detection algorithm is obtained, the algorithm updater (personnel) can publish the upgraded version of the anomaly detection algorithm, such as uploading it to a server, so that each vehicle-mounted video recording device can obtain the upgraded version of the anomaly detection algorithm by downloading, receiving, etc., so as to upgrade the local anomaly detection algorithm, thereby improving the accuracy of the local anomaly detection algorithm in identifying abnormal behaviors or abnormal situations such as abnormal driving behaviors, dangerous vehicle conditions, etc., and further improving the vehicle assisted driving function.

[0053] An image acquisition method provided by an embodiment of the present invention obtains an image data set collected by a vehicle-mounted camera; performs anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type; obtains a target image data set from the image data set based on the confidence level; determines the label information of the target image data set based on the anomaly detection type, and stores the target image data frame in the target image data set based on the label information for training the anomaly detection algorithm. The method obtains the target image data set based on the confidence level after obtaining the anomaly detection type and the confidence level of the anomaly detection type, and determines the label information of the target image data set based on the anomaly detection type, so that the target of obtaining the target image data set is clear, and no manual screening and classification is required, thereby improving the efficiency, cost and effectiveness of obtaining the target image data set; by automatically classifying and storing the target image data frame based on the label information, targeted training samples are conveniently provided for training the anomaly detection algorithm, avoiding the blindness of the training of the anomaly detection algorithm, shortening the iterative upgrade cycle of the anomaly detection algorithm, and improving the efficiency of improving the recognition effect of the anomaly detection algorithm.

[0054] As a first optional embodiment of this embodiment, based on the above embodiment, the step of obtaining the target image dataset from the image dataset according to the confidence level may be specifically implemented as the following steps:

[0055] a1) If the confidence level is within a preset confidence interval, a preset number corresponding to the confidence level of the confidence level is obtained.

[0056] The preset confidence interval can be understood as a preset confidence interval for determining whether to acquire the target image data set. The preset number can be understood as a preset number corresponding to the target image data frames constituting the target image data set.

[0057] It should be noted that the preset confidence interval can be set according to actual needs and / or the confidence threshold of the anomaly detection type output by the anomaly detection algorithm. For example, if the confidence threshold of the anomaly detection type output by the anomaly detection algorithm is set to 0.6, that is, when the anomaly detection type is detected and the corresponding confidence of the type exceeds 60%, the anomaly detection algorithm will output the anomaly detection type and the confidence of the anomaly detection type. At this time, the lower limit of the confidence interval set should be above 0.6. According to actual needs, if you want to obtain a target image data set with high reliability and a certain degree of suspicion, you can set the confidence interval to [0.75, 0.95].

[0058] Continuing with the above description, a plurality of confidence levels can be pre-divided in the confidence interval according to the confidence from low to high, that is, the higher the confidence level, the more reliable the anomaly detection type output by the anomaly detection algorithm is. For example, if the confidence interval is [0.7, 0.9], the confidence of [0.7, 0.75] can be classified as the lowest level, (0.75, 0.8] as the second lowest level, (0.8, 0.85] as the medium level, and (0.85, 0.9] as the higher level. At the same time, a preset number of target image data frames can be allocated in advance for each confidence level, and the preset numbers corresponding to each confidence level can be the same or different, and can be a preset fixed value or can be adaptively adjusted with the number of confidence level divisions.

[0059] In this embodiment, if the obtained confidence of the abnormality detection type is within a preset confidence interval, a confidence level corresponding to the confidence is determined, and a preset number corresponding to the confidence level is obtained.

[0060] As one implementation manner, the confidence level is inversely proportional to the preset number.

[0061] In this embodiment, the confidence level is inversely proportional to the preset number, that is, when the confidence level is higher, the output abnormality detection type is considered to be more reliable, that is, the abnormality detection algorithm is more accurate in detecting the current abnormal behavior or abnormal situation, so fewer target image data frames can be obtained; and when the confidence level is lower, it means that the abnormality detection algorithm is not good at judging the current abnormal behavior or abnormal situation, so more target image data frames can be obtained to provide more diverse abnormal situations.

[0062] Exemplarily, the confidence interval is divided into N confidence levels, L1, L2, ..., LN from low to high, and the preset number corresponding to each confidence level can be determined as N, N-1, ..., 1, respectively.

[0063] The above technical solution of this embodiment sets the confidence level inversely proportional to the preset number, and obtains fewer target image data frames at a high confidence level. On the basis of ensuring the training quality of the anomaly detection algorithm, it reduces data redundancy, saves storage space and computing resources, improves the training and updating efficiency and reliability of the anomaly detection algorithm, and better avoids the anomaly detection algorithm from overfitting normal data and ignoring abnormal data; more target image data frames are obtained at a low confidence level, ensuring that there is sufficiently diverse data to optimize the anomaly detection algorithm, so as to improve the generalization ability and accuracy of the anomaly detection algorithm.

[0064] b1) Acquire a target image data set from the image data set according to the preset number.

[0065] In this embodiment, the image data set in which the target image data frame corresponding to the abnormality detection type is located is determined, and a preset number of target image data frames are obtained from the image data set to form the target image data set. The specific method of obtaining the preset number of target image data frames from the image data set may be to obtain a target image data frame after a certain number of frames or a certain time after the target image data frame corresponding to the abnormality detection type in the image data set, until the number of acquired target image data frames reaches the preset number; or to obtain target image data frames adjacent to the target image data frame corresponding to the abnormality detection type in the image data set before and after the target image data frame respectively at a certain number of frames or time intervals, until the total number of acquired target image data frames reaches the preset number.

[0066] The above technical scheme of this embodiment ensures the diversity and effectiveness of the target image data set by determining the preset number of target image data frames that need to be acquired to constitute the target image data set according to the confidence level corresponding to the confidence when the acquired confidence is within the preset confidence interval, and acquiring the target image data set from the image data set according to the preset number, thereby providing strong support for the subsequent training of the anomaly detection algorithm.

[0067] As one implementation manner, this optional embodiment may further specify the step of obtaining the target image dataset from the image dataset according to the preset number as follows:

[0068] b11) determining an abnormal image data frame corresponding to the abnormality detection type from the image data set, wherein each image data frame in the image data set is arranged in the order of acquisition timestamps.

[0069] The abnormal image data frame may be understood as an image data frame that is considered by an abnormal detection algorithm to have abnormal behavior or abnormal situation.

[0070] In this embodiment, the abnormal image data frame corresponding to the acquired abnormality detection type may be determined from the image data set according to information such as the number of the image data frame.

[0071] b12) acquiring a preset number of target image data frames after the abnormal image data frame at a preset time interval to form a target image data set.

[0072] In this embodiment, a unique time interval for acquiring each target image data frame can be preset based on experience, or the time interval for acquiring each target image data frame can be set separately for different abnormal behaviors or abnormal situations. For example, a longer time interval is set for abnormal behaviors with a long duration such as making a phone call, such as acquiring a target image data frame every 1 second, and a shorter time interval is set for abnormal behaviors with a short duration such as smoking, such as acquiring a target image data frame every 0.2 seconds.

[0073] Following the above description, a preset number of target image data frames are acquired at preset time intervals after the abnormal image data frame, and a target image data set is formed based on the acquired target image data frames.

[0074] The above technical solution of this embodiment avoids the existence of repeated target image data frames in the target image data set by determining the abnormal image data frame and obtaining a preset number of target image data frames after the abnormal image data frame at a preset time interval, and obtains the target image data frame adjacent to the abnormal image data frame, thereby further improving the effectiveness of the obtained target image data frame.

[0075] As a second optional embodiment of this embodiment, the step of determining the label information of the target image data set according to the abnormality detection type and storing the target image data frames in the target image data set based on the label information may be specifically optimized into the following steps:

[0076] a2) using the abnormality detection type as label information of the target image data set, wherein the abnormality detection type includes an abnormal driving behavior type or a dangerous vehicle condition type.

[0077] In this embodiment, the abnormal driving behavior types may include smoking, fatigue, making phone calls and / or distraction, etc., and the dangerous vehicle condition types may include too close distance, lane departure, road fault and / or pedestrian crossing, etc. The abnormal detection type is used as the label information of the target image data set, that is, each target image data frame in the target image data set will be marked as the abnormal detection type, which is equivalent to each target image data frame having label information consistent with the target image data set.

[0078] b2) Obtain the storage classification path corresponding to the tag information.

[0079] In this embodiment, a corresponding storage classification path is defined in advance for each type of tag information, and the storage classification path may be a classification table path in a server database or a directory path in a server file system. The storage classification path corresponding to the tag information is obtained by determining the tag information of the target image data set or traversing the tag information of each target image data frame in the target image data set.

[0080] c2) uploading the target image data frame in the target image data set to the server according to the storage classification path.

[0081] In this embodiment, each target image data frame in the target image data set is uploaded to the corresponding path of the server according to the storage classification path. The uploading method can adopt the file transfer protocol (FTP), secure file transfer protocol (SFTP), HTTP / HTTPS, and cloud storage service API, etc. Each target image data frame can be compressed and processed before uploading.

[0082] It is understandable that the server has been pre-configured with the corresponding storage classification path and has the authority to receive the uploaded target image data frame.

[0083] The above technical solution of this embodiment, by taking the anomaly detection type as the label information of the target image data set, uploads the target image data frame to the server according to the storage classification path corresponding to the label information, thereby ensuring the orderly storage and efficient access of the target image data frame, saving manpower and time costs, and being able to provide more targeted training samples for the anomaly detection algorithm, avoiding the blindness of the anomaly detection algorithm training, thereby effectively improving the performance of the anomaly detection algorithm and accelerating the iterative upgrade of the anomaly detection algorithm.

[0084] As one implementation manner, this optional embodiment may further concretize the uploading of the target image data frame in the target image data set to the server according to the storage classification path as follows:

[0085] c21) determining the image quality level of each target image data frame in the target image data set, and removing the target image data frames whose image quality levels do not meet the preset level conditions from the target image data set.

[0086] In this embodiment, the standard of image quality level is predefined, and the standard may include resolution (pixel), clarity, brightness, contrast, etc. For example, only clarity may be used as the standard, that is, the image quality levels corresponding to the four clarity levels of very blurry, generally blurry, generally clear, and very clear are respectively determined as low, second low, medium, and high. Then, the level conditions may be preset as needed, such as the level conditions may be second low or medium.

[0087] In this embodiment, after the target image data set is acquired, an image processing tool such as OpenCV can be used to evaluate and determine the image quality level of each target image data frame in the target image data set in real time. If it is determined that the image quality level does not meet the preset level condition, the target image data frame corresponding to the image quality level is removed from the target image data set. Exemplarily, following the above example description, if the preset level condition is the second lowest, each target image data frame with a low image quality level is removed from the target image data set, and if the preset level condition is medium, each target image data frame with a low and second lowest image quality level is removed from the target image data set.

[0088] It should be noted that if the target image data set also includes a video consisting of continuous target image data frames, the image quality level of the video can be determined at the same time. If the preset level condition is not met, the video will also be removed from the target image data set.

[0089] c22) obtaining a network traffic consumption value of the vehicle-mounted video recording device, and if the network traffic consumption value is greater than a set traffic threshold, determining an upload priority of each target image data frame in the target image data set.

[0090] It can be understood that the rules for determining the upload priority are set in advance, and the upload priority can be based on factors such as image quality level, abnormality detection type and / or adjacent distance (time or number of frames) to the abnormal image data frame. For example, the upload priority can be determined according to the image quality level, and the target image data frame with a higher image quality level has a higher corresponding upload priority.

[0091] In this embodiment, the network traffic consumption value of the vehicle-mounted video recording device is obtained in real time or when the vehicle-mounted video recording device needs to send the target image data frame through a network traffic monitoring tool or API, wherein the network traffic consumption value may be the historical consumption value of the day, the historical consumption value of the month, or the historical consumption value within any set time. If the currently acquired network traffic consumption value is less than the set traffic threshold, it is not necessary to determine the upload priority of each target image data frame, and the target image data frame can be sent in sequence according to the default sorting; if the current network traffic consumption value is greater than the set traffic threshold, in order to avoid excessive traffic consumption or terminal offline within a specific time, the upload priority of each target image data frame in the target image data set is determined according to the preset rules.

[0092] c23) uploading the target image data frame to the server according to the upload priority of the target image data frame and the storage classification path, until all the target image data frames in the target image data set are uploaded or the network traffic consumption value reaches a critical value.

[0093] In this embodiment, the target image frame can be uploaded to the server according to the upload priority of the target image data frame and the storage classification path through multi-threaded or asynchronous uploading methods. During the uploading process, the network traffic consumption value is continuously monitored. If the network traffic consumption value reaches a critical value, the uploading is stopped to ensure that the traffic limit is not exceeded.

[0094] The above technical scheme of this embodiment ensures the validity of the uploaded target image data frames and reduces the waste of computing resources by removing low-quality target image data frames; ensures that the traffic limit will not be exceeded by monitoring the network traffic consumption value of the vehicle-mounted video recording device, thereby avoiding the vehicle-mounted video recording device from being offline or incurring additional costs; and when the network traffic consumption value is greater than the set traffic threshold, determines the upload priority of each target image data frame, and uploads all target image data frames to the server according to the upload priority and the storage classification path, ensuring that more important target image data frames are uploaded first.

[0095] Figure 2 FIG. 1 is a schematic diagram of the structure of an image acquisition device provided by an embodiment of the present invention. Figure 2 As shown, the device includes: a first image data set acquisition module 21, an abnormality detection module 22, a second image data set acquisition module 23 and an image data set storage module 24, wherein:

[0096] A first image data set acquisition module 21, used to acquire an image data set collected by a vehicle-mounted camera;

[0097] An anomaly detection module 22, used to perform anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type;

[0098] A second image data set acquisition module 23, configured to acquire a target image data set from the image data set according to the confidence level;

[0099] The image data set storage module 24 is used to determine the label information of the target image data set according to the anomaly detection type, and store the target image data frames in the target image data set based on the label information for training the anomaly detection algorithm.

[0100] An image acquisition device provided by an embodiment of the present invention obtains an image data set collected by a vehicle-mounted camera; performs anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type; obtains a target image data set from the image data set based on the confidence level; determines the label information of the target image data set based on the anomaly detection type, and stores the target image data frame in the target image data set based on the label information for training the anomaly detection algorithm. The method obtains the target image data set based on the confidence level after obtaining the anomaly detection type and the confidence level of the anomaly detection type, and determines the label information of the target image data set based on the anomaly detection type, so that the target of obtaining the target image data set is clear, and no manual screening and classification is required, thereby improving the efficiency, cost and effectiveness of obtaining the target image data set; by automatically classifying and storing the target image data frame based on the label information, targeted training samples are conveniently provided for training the anomaly detection algorithm, avoiding the blindness of the training of the anomaly detection algorithm, shortening the iterative upgrade cycle of the anomaly detection algorithm, and improving the efficiency of improving the recognition effect of the anomaly detection algorithm.

[0101] Furthermore, the second image data set acquisition module 23 may specifically include:

[0102] a preset number acquisition unit, configured to acquire a preset number corresponding to a confidence level of the confidence level if the confidence level is within a preset confidence interval, wherein the confidence level is inversely proportional to the preset number;

[0103] The third image data set acquisition unit is configured to acquire a target image data set from the image data sets according to the preset number.

[0104] Furthermore, the third image data set acquisition unit may be specifically used for:

[0105] Determining an abnormal image data frame corresponding to the abnormality detection type from the image data set, wherein each image data frame in the image data set is arranged in order of acquisition timestamps;

[0106] A preset number of target image data frames are acquired after the abnormal image data frame at a preset time interval to form a target image data set.

[0107] Furthermore, the image data set storage module 24 may specifically include:

[0108] a label information determination unit, configured to use the abnormality detection type as label information of a target image data set, wherein the abnormality detection type includes an abnormal driving behavior type or a dangerous vehicle condition type;

[0109] A storage classification path determining unit, used to obtain the storage classification path corresponding to the tag information;

[0110] An image uploading unit is used to upload the target image data frame in the target image data set to a server according to the storage classification path.

[0111] Furthermore, the image uploading unit may specifically include:

[0112] Determining the image quality level of each of the target image data frames in the target image data set, and removing the target image data frames whose image quality levels do not meet the preset level conditions from the target image data set;

[0113] Obtaining a network traffic consumption value of the vehicle-mounted video recording device, and if the network traffic consumption value is greater than a set traffic threshold, determining an upload priority of each target image data frame in the target image data set;

[0114] According to the upload priority of the target image data frame and the storage classification path, the target image data frame is uploaded to the server until all the target image data frames in the target image data set are uploaded or the network traffic consumption value reaches a critical value.

[0115] The image acquisition device provided in the embodiment of the present invention can execute the image acquisition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Figure 3A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 3 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0118] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 31 performs the various methods and processes described above, such as an image acquisition method.

[0120] In some embodiments, the image acquisition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the image acquisition method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to perform the image acquisition method in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image acquisition method, characterized in that: Applied to vehicle video recording equipment, including: Obtain image datasets collected by vehicle-mounted cameras; Performing anomaly detection on the image data set based on an anomaly detection algorithm to obtain an anomaly detection type and a confidence level of the anomaly detection type; Acquire a target image dataset from the image dataset according to the confidence level; The label information of the target image data set is determined according to the anomaly detection type, and the target image data frames in the target image data set are stored based on the label information for training the anomaly detection algorithm.

2. The method according to claim 1, characterized in that: The acquiring a target image dataset from the image dataset according to the confidence level includes: If the confidence level is within a preset confidence interval, obtaining a preset number corresponding to the confidence level of the confidence level; A target image data set is acquired from the image data set according to the preset number.

3. The method according to claim 2, characterized in that The confidence level is inversely proportional to the preset number.

4. The method according to claim 2, characterized in that: The acquiring the target image data set from the image data set according to the preset number includes: Determining an abnormal image data frame corresponding to the abnormality detection type from the image data set, wherein each image data frame in the image data set is arranged in order of acquisition timestamps; A preset number of target image data frames are acquired after the abnormal image data frame at a preset time interval to form a target image data set.

5. The method according to claim 1, characterized in that The step of determining label information of the target image data set according to the abnormality detection type, and storing the target image data frame in the target image data set based on the label information, comprises: The abnormality detection type is used as label information of the target image data set, and the abnormality detection type includes an abnormal driving behavior type or a dangerous vehicle condition type; Obtaining a storage classification path corresponding to the tag information; The target image data frame in the target image data set is uploaded to the server according to the storage classification path.

6. The method according to claim 5, characterized in that The uploading the target image data frame in the target image data set to the server according to the storage classification path includes: Determining the image quality level of each of the target image data frames in the target image data set, and removing the target image data frames whose image quality levels do not meet the preset level conditions from the target image data set; Obtaining a network traffic consumption value of the vehicle-mounted video recording device, and if the network traffic consumption value is greater than a set traffic threshold, determining an upload priority of each target image data frame in the target image data set; According to the upload priority of the target image data frame and the storage classification path, the target image data frame is uploaded to the server until all the target image data frames in the target image data set are uploaded or the network traffic consumption value reaches a critical value.

7. An image acquisition device, characterized in that: Applied in vehicle video recording equipment, including: A first image data set acquisition module, used to acquire an image data set collected by a vehicle-mounted camera; An anomaly detection module, used to perform anomaly detection on the image data set based on an anomaly detection algorithm, and obtain an anomaly detection type and a confidence level of the anomaly detection type; a second image data set acquisition module, configured to acquire a target image data set from the image data set according to the confidence level; The image data set storage module is used to determine the label information of the target image data set according to the anomaly detection type, and store the target image data frames in the target image data set based on the label information for training the anomaly detection algorithm.

8. An electronic device, characterized in that: As the vehicle-mounted video recording device according to claim 7, the electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image acquisition method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image acquisition method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image acquisition method according to any one of claims 1 to 6.

Images