An image recognition method and related equipment based on artificial intelligence
Through the image recognition method based on artificial intelligence, the image collected by the classification model is crowdsourced by the identification device, and the problem of low manual recognition efficiency is solved, the image recognition is automated and efficient and accurate, and the quality of image data is improved.
Patent Information
- Application Number
- CN201911008432.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-10-22
AI Technical Summary
In the prior art, image recognition mainly relies on manual methods, resulting in low recognition efficiency and inaccurateness. Especially in the image data collected by the equipment crowdsourcing, there are a large number of unavailable images, which affects the quality of subsequent jobs.
Using an artificial intelligence-based image recognition method, by obtaining the image set to be identified and traversing the images therein, using a classification model to identify unavailable image types such as backlight, dimness, object occlusion, reflective occlusion and tilt types, automatically identifying and determining whether the image is Available images.
The automation and efficient accuracy of image recognition are achieved, the accuracy of image recognition is improved, manual intervention is reduced, and the quality and efficiency of image data are improved.
Smart Images

Figure CN111738282B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to an image recognition method and related equipment based on artificial intelligence. Background Art
[0002] In many scenarios, it is necessary to identify qualified images from a large number of images of different quality levels as usable images. Then, other subsequent operations are performed based on these usable images. For example, in model training, it is usually necessary to select high-quality images from a large number of images of different quality levels as training data.
[0003] Currently, the recognition is mainly done manually, that is, identifying usable images from images of different quality levels by naked eyes. Summary of the invention
[0004] In order to solve the above technical problems, the present application provides an image recognition method and related equipment based on artificial intelligence, which can efficiently and accurately determine whether the image to be recognized is a usable image.
[0005] The embodiments of the present application disclose the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides an image recognition method based on artificial intelligence, the method comprising:
[0007] Acquire a set of images to be identified, wherein the set of images to be identified includes at least two images to be identified;
[0008] Traversing the images to be identified in the set of images to be identified, and determining whether the images to be identified belong to an unusable image type according to a classification model, wherein the unusable image type includes any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type;
[0009] If not, it is determined that the image to be identified is an available image.
[0010] In a second aspect, an embodiment of the present application provides an image recognition device based on artificial intelligence, the device comprising an acquisition unit and a determination unit:
[0011] The acquisition unit is used to acquire a set of images to be recognized, wherein the set of images to be recognized includes at least two images to be recognized;
[0012] The determination unit is used to traverse the images to be identified in the set of images to be identified, and determine whether the images to be identified belong to an unusable image type according to the classification model, where the unusable image type includes any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type;
[0013] The determining unit is further configured to determine, if not, whether the image to be identified is an available image.
[0014] In a third aspect, an embodiment of the present application provides an image recognition device based on artificial intelligence, the device comprising a processor and a memory:
[0015] The memory is used to store program code and transmit the program code to the processor;
[0016] The processor is used to execute the artificial intelligence-based image recognition method as described in the first aspect according to the instructions in the program code.
[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the artificial intelligence-based image recognition method as described in the first aspect.
[0018] It can be seen from the above technical solution that after obtaining a set of images to be identified including at least two images to be identified, the images to be identified in the set can be automatically traversed, and the classification model can be used to determine whether the image to be identified belongs to an unusable image type, wherein the unusable image type can include any one or more combinations of backlight type, dim type, object occlusion type, reflective occlusion type, and tilt type. If the image to be identified does not belong to an unusable image type, it can be determined that the image to be identified is an available image, i.e., a high-quality image. In this way, it can be efficiently and accurately determined whether the image to be identified is an available image. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1 A schematic diagram of an application scenario of an image recognition method based on artificial intelligence provided in an embodiment of the present application;
[0021] Figure 2 A flowchart of an image recognition method provided in an embodiment of the present application;
[0022] Figure 3 An image schematic diagram of an unavailable image type provided in an embodiment of the present application;
[0023] Figure 4A flowchart of a specific implementation method of image recognition provided in an embodiment of the present application;
[0024] Figure 5 is a flow chart of a method for manually checking an image;
[0025] Figure 6 An image schematic diagram of another unavailable image type provided in an embodiment of the present application;
[0026] Figure 7 A flowchart of a specific implementation method of image recognition provided in an embodiment of the present application;
[0027] Figure 8 An image schematic diagram of crowdsourcing image data of an interception device provided in an embodiment of the present application;
[0028] Fig. 9 A schematic diagram of a convolution kernel provided in an embodiment of the present application;
[0029] Fig.10 A schematic diagram of an image to be identified provided in an embodiment of the present application;
[0030] Fig.11 An overall flow chart of an image recognition method based on artificial intelligence provided in an embodiment of the present application;
[0031] Fig.12 A structural diagram of a cascaded intelligent recognition system provided in an embodiment of the present application;
[0032] Fig.13 An artificial intelligence-based image recognition device provided in an embodiment of the present application;
[0033] Fig.14 A structural diagram of an artificial intelligence-based image recognition device provided in an embodiment of the present application;
[0034] Fig.15 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0035] The embodiments of the present application are described below in conjunction with the accompanying drawings.
[0036] Currently, recognition is mainly done manually, that is, identifying usable images from images of different quality levels by naked eyes, which results in a low image recognition rate.
[0037] To this end, the embodiment of the present application provides an image recognition method based on artificial intelligence, so that after obtaining the set of images to be recognized, it is possible to realize automatic traversal of the images to be recognized, and identify whether the images to be recognized belong to the type of unavailable images according to the classification model, thereby determining whether the images to be recognized are available images. Based on this automatic traversal method and intelligent recognition method, it is possible to efficiently and accurately determine whether the images to be recognized are available images.
[0038] The image recognition method provided in the embodiments of the present application and the corresponding classification model training method can all be implemented based on artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.
[0039] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0040] In the embodiments of the present application, the computer vision technology direction in artificial intelligence software technology is mainly involved.
[0041] For example, it may involve Image Semantic Understanding (ISU) in Computer Vision technology, including image classification, image feature extraction, etc.
[0042] The image recognition method provided in the present application can be applied to image processing devices, such as terminal devices and servers. The terminal device can be, for example, an intelligent terminal, a computer, a personal digital assistant (PDA), a tablet computer, and the like. The image recognition method can also be applied to a server, which can be an independent server or a server in a cluster.
[0043] The image processing device may also have the ability to implement computer vision technology. Computer vision is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.
[0044] In the embodiment of the present application, the image processing device can identify and classify the image to be identified through computer vision technology.
[0045] To facilitate understanding of the technical solution of the present application, the artificial intelligence-based image recognition method provided in the embodiment of the present application is introduced below in combination with actual application scenarios.
[0046] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of an image recognition method based on artificial intelligence provided in an embodiment of the present application. Figure 1 As shown, the scene includes a server 101, and the server 101 can execute the image recognition method provided by the embodiment of the present application. A classification model is deployed in the server 101, and the classification model can realize the function of classifying the input image to be recognized. Among them, the classification type of image classification performed by the classification model is an unavailable image type. The unavailable image type may include any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type.
[0047] In the embodiment of the present application, the server 101 may obtain a set of images to be identified, wherein the set of images to be identified may include at least two images to be identified.
[0048] Then, the server 101 can traverse the images to be identified in the set of images to be identified. For an image to be identified, the server 101 can determine whether the image to be identified belongs to an unavailable image type according to the classification model deployed therein. If not, the server 101 can determine that the image to be identified is an available image.
[0049] The method can efficiently and accurately determine whether the image to be identified is a usable image through an automated traversal method and an intelligent recognition method.
[0050] Next, taking the server as the above-mentioned image processing device as an example, the image recognition method provided in the embodiment of the present application is introduced in combination with the accompanying drawings.
[0051] See also Figure 2 , which shows a flow chart of an image recognition method provided by an embodiment of the present application, the method comprising:
[0052] S201: Obtain a set of images to be identified.
[0053] The set of images to be recognized may include at least two images to be recognized.
[0054] The embodiment of the present application does not limit the method for obtaining the image set to be identified, and a suitable method can be selected to obtain the image set to be identified according to the actual situation. For example, a user can send the image set to be identified to a server, and the server obtains the image set to be identified.
[0055] S202: Traverse the images to be identified in the image set to be identified, and determine whether the images to be identified belong to an unavailable image type according to the classification model. If not, execute S203.
[0056] In the embodiment of the present application, the unavailable image type may include any one or more combinations of backlight type, dim type, object occlusion type, reflective occlusion type, and tilt type. Among them, the backlight type image may refer to an image taken when the subject is between the light source and the camera. The reflective occlusion type image may be an image displayed when the subject is blocked by the reflective image of other objects.
[0057] See also Figure 3 , which shows an image schematic diagram of an unavailable image type provided by an embodiment of the present application, such as Figure 3 As shown, the figure shows images of backlight type, dim type, object occlusion type, reflective occlusion type and tilted type respectively.
[0058] The classification model can be obtained based on deep learning convolutional neural network model training (such as Inception v3), and the classification model can identify and classify the image to be identified. After the image to be identified is input into the classification model, the classification model can output the type of the image to be identified. If the classification model identifies that the image to be identified belongs to an unavailable image type, it can output the specific type result of the image to be identified belonging to the unavailable image type. If the classification model cannot identify whether the image to be identified belongs to an unavailable image type, it can output the recognition result of the image to be identified corresponding to the uncertain image. If the classification model identifies that the image to be identified does not belong to an unavailable image type, it can output the recognition result that the image to be identified belongs to an available image.
[0059] S203: Determine whether the image to be identified is an available image.
[0060] The specific implementation of S201-S203 is introduced below. Figure 4 , which shows a flowchart of a specific implementation method of image recognition provided by an embodiment of the present application. Figure 4 As shown, the classification model 1 can identify whether an image belongs to unusable image type 1, and the unusable image type 1 can include any one or more combinations of backlight type, dim type, object occlusion type, reflective occlusion type and tilt type. For the image set to be identified, the images to be identified are input into the classification model 1 in sequence. Then, it can be identified whether each image belongs to unusable image type 1. If it is determined that the image belongs to unusable image type 1, the image is determined to be an unusable image. If it is determined that the image does not belong to unusable image type 1, the image can be determined to be an available image.
[0061] It can be seen from the above technical solution that after obtaining a set of images to be identified including at least two images to be identified, the images to be identified in the set can be automatically traversed, and the classification model can be used to determine whether the image to be identified belongs to an unusable image type, wherein the unusable image type can include any one or more combinations of backlight type, dim type, object occlusion type, reflective occlusion type, and tilt type. If the image to be identified does not belong to an unusable image type, it can be determined that the image to be identified is an available image, i.e., a high-quality image. In this way, it can be efficiently and accurately determined whether the image to be identified is an available image.
[0062] At present, in order to ensure a better user experience, map products need to update map content in a timely and accurate manner. Therefore, the content must be updated regularly or frequently. For example, if a new road is opened, this information needs to be updated to the map content library in a timely manner. In fact, map manufacturers have multiple ways to collect new map information, and the user crowdsourcing collection model has become the most important method of map data collection due to its high efficiency and low cost.
[0063] At present, user crowdsourcing can be divided into two modes: manual crowdsourcing (active collection) and device crowdsourcing (passive collection). The device crowdsourcing collection mode is: all devices that have installed the data collection software development kit (SDK) are a device for crowdsourcing collection in this mode. Compared with manual crowdsourcing, device crowdsourcing has lower costs, higher data return efficiency, and larger data collection volume. It is becoming the mainstream user crowdsourcing collection method.
[0064] However, equipment crowdsourcing is equivalent to manual crowdsourcing without the high requirements of data collection (such as placement, lens pitch angle, collection time), etc., and much of the data sent back is unusable, such as images collected against light (resulting in unclear details in the image), lens tilt (resulting in incomplete collection of road elements), physical objects blocking the lens (resulting in incomplete collection of road elements), reflections from objects in the car (resulting in unclear details of elements in the image), images collected or sent back at night are very dark (resulting in unclear details in the image), etc. This results in high costs for sending back data and seriously affects the quality of subsequent operations based on the images.
[0065] For this question, see Figure 5 , which shows a flow chart of a method for manually checking images in the related art. Figure 5 As shown in the figure, the image data collected by crowdsourcing application equipment is mainly checked manually to filter out images with problems and identify images that are acceptable for subsequent operations. Since each image is checked by the naked eye, the efficiency is low; and each inspector has a different understanding of the inspection rules, which may lead to inconsistent inspection results; and adding manual inspection will result in a waste of manpower and material resources.
[0066] Based on the above problem, in a possible implementation, the device crowdsourced image data can be used as the image to be identified in the set of images to be identified in S201. Thus, it can be efficiently and accurately determined whether the device crowdsourced image data is an available image.
[0067] In actual scenarios, for device crowdsourced image data, images may be unavailable due to the following reasons, such as images collected when the vehicle is stopped (resulting in a large part of the image being without roads) and images collected on rainy days (resulting in the road being blocked by rain in the image).
[0068] Based on this, in order to improve the accuracy of the classification model in S202 in identifying the usability of the image to be identified, in a possible implementation, the unusable image type in S202 also includes one or more combinations of non-road type and non-sunny day type. Figure 6, which shows an image schematic diagram of another unavailable image type provided by an embodiment of the present application, such as Figure 6 As shown, the figure shows images of non-road type and non-sunny type respectively.
[0069] That is to say, the classification model of the server can also identify whether the image to be identified belongs to the non-road type or the non-sunny type. If it is identified that the image to be identified belongs to either of these two types, it can be determined that the image to be identified is not an available image.
[0070] The specific implementation of S201-S203 is introduced below. Figure 7 , which shows a flowchart of a specific implementation method of image recognition provided by an embodiment of the present application. Figure 7 As shown, a classification model 2 may also be trained so that it can achieve the following functions: identify the input image to be identified and determine whether it is a non-road type or non-sunny type image.
[0071] In a specific implementation, after the classification model 1 identifies whether the image to be identified belongs to the unavailable image type 1, the image to be identified can also be input into the classification model 2 to identify whether the image to be identified belongs to the non-road type and / or the non-sunny type. If it is determined that the image to be identified belongs to any one or both of these two types, it can be determined that the image to be identified belongs to the unavailable image type, that is, it is determined that the image to be identified is an unavailable image. If it is determined that the image to be identified does not belong to any one or both of these two types, it can be determined that the image to be identified does not belong to the unavailable image type, that is, it is determined that the image to be identified is an available image.
[0072] As a result, the usability recognition accuracy of the classification model for the image to be recognized is improved.
[0073] In actual scenarios, the image to be recognized may be blurred (ie, the details in the image are not clear). Such an image should be avoided from being determined as a usable image in S203.
[0074] To this end, in a possible implementation, the method of determining in S203 that the image to be identified is a usable image may include:
[0075] S301: Determine whether the image to be recognized is a blurred image. If not, execute S302.
[0076] S302: Determine whether the image to be identified is an available image.
[0077] In an embodiment of the present application, after the classification model identifies the image to be identified as a usable image, it can also be determined whether the image to be identified is a blurred image. If not, it can be determined that the image to be identified is a usable image.
[0078] Through this method, the usability recognition accuracy of the image to be recognized can be further improved.
[0079] In a possible implementation, the method of determining whether the image to be identified is a blurred image in S301 may include:
[0080] S401: intercepting a first target area from the image to be identified, where the first target area is a non-sky area.
[0081] In fact, when identifying whether an image is blurred based on the sky area in the image, the determination result obtained may be wrong (i.e., determining that the image is blurred). To this end, in an embodiment of the present application, a first target area that is not a sky area may be intercepted from the image to be identified.
[0082] See also Figure 8 , which shows an image schematic diagram of a crowdsourcing image data interception device provided by an embodiment of the present application. Figure 8 As shown, when capturing the first target area of the crowdsourced image data of the device, two centrally symmetrical fixed areas can be captured along the horizontal direction (x direction) of the image, and then the two fixed areas are merged into a complete area as the first target area.
[0083] S402: Generate a grayscale image corresponding to the target area.
[0084] S403: Convolve the grayscale image according to the convolution kernel to generate a target grayscale image.
[0085] In the embodiment of the present application, a convolution kernel can be set, see Fig. 9 , which shows a schematic diagram of a convolution kernel provided in an embodiment of the present application, such as Fig. 9 As shown, the kernel is a 3×3 Laplacian kernel. Then, the grayscale image can be convolved according to the convolution kernel to generate a target grayscale image.
[0086] S404: Calculate the grayscale variance of all pixels in the target grayscale image.
[0087] The grayscale variance of all the pixels in the target grayscale image can be determined based on the grayscale value of each pixel in the target grayscale image.
[0088] S405: Determine whether the grayscale variance is less than a preset variance threshold, if so, determine that the image to be identified is a blurred image, if not, determine that the image to be identified is not a blurred image.
[0089] A variance threshold may be preset, and the variance threshold is used to identify whether the image is a blurred image. If the grayscale variance of the image is less than the variance threshold, the image may be identified as a blurred image. If the grayscale variance of the image is not less than the variance threshold, the image may be identified as not a blurred image.
[0090] In the embodiment of the present application, if the server determines that the grayscale variance is less than a preset variance threshold, it is determined that the image to be identified is a blurred image.
[0091] Through this method, a blurred image to be identified can be identified to avoid identifying it as a usable image, thereby further improving the recognition accuracy of the image to be identified.
[0092] In actual scenes, the image to be identified may have a too-biased shooting angle, such as a too-large or too-small shooting pitch angle, which may result in incomplete collection of (road) elements in the image to be identified. To this end, in a possible implementation, the method of determining that the image to be identified is a usable image in S203 may include:
[0093] S501: Determine whether the shooting angle of view corresponding to the image to be identified meets a preset angle of view range. If yes, execute S502.
[0094] S502: Determine whether the image to be identified is an available image.
[0095] A viewing angle range can be preset. If the viewing angle range of the image meets the range, it can be identified that the shooting angle corresponding to the image is within the normal range, that is, the image includes relatively comprehensive elements. If the shooting angle of the image does not meet the range, it is identified that the shooting angle corresponding to the image is not within the normal range, that is, the elements included in the image are not comprehensive.
[0096] In an embodiment of the present application, the server may determine whether the shooting angle of view corresponding to the image to be identified meets a preset angle of view range. If so, it may be determined that the image to be identified is an available image.
[0097] In a possible implementation, the method of determining in S501 whether the shooting angle of view corresponding to the image to be recognized satisfies a preset angle of view range may include:
[0098] S601: Divide a second target area corresponding to a preset viewing angle range from the image to be recognized.
[0099] The following uses crowdsourced image data from a device as an example of an image to be identified to illustrate a method for determining whether a shooting angle corresponding to the image to be identified satisfies a preset angle range.
[0100] In the embodiment of the present application, the image to be recognized may be divided to obtain a second target area corresponding to a preset viewing angle range.
[0101] See also Fig.10 , which shows a schematic diagram of an image to be recognized provided by an embodiment of the present application, such as Fig.10 As shown, the image to be identified can be divided into 9 areas of 3×3 in the horizontal direction, and areas 1, 2 and 3 in the image are used as areas corresponding to "the shooting angle of view is too downward"; areas 7, 8 and 9 in the image are used as areas corresponding to "the shooting angle of view is too upward"; areas 1, 4 and 7 in the image are used as areas corresponding to "the shooting angle of view is too left-biased"; areas 3, 6 and 9 in the image are used as areas corresponding to "the shooting angle of view is too right-biased"; and area 5 in the image is used as the second target area corresponding to "the shooting angle of view meets the preset angle of view range".
[0102] S602: Determine the target position of the target object in the image to be recognized.
[0103] In an embodiment of the present application, the target object in the device crowdsourced image data may be a road vanishing point, wherein the road vanishing point may refer to a point where two or more parallel lines extend toward the distant horizon until they converge.
[0104] First, the lane line or road boundary can be detected, then the road vanishing point can be estimated according to the lane line or road boundary, and finally, the target position of the road vanishing point in the image to be identified can be determined.
[0105] based on Fig.10 In the corresponding example, it is determined that the target object, the road vanishing point, is located in the area 8 (target position) in the image to be recognized.
[0106] S603: Determine whether the target position is in the second target area. If yes, execute S603.
[0107] S604: Determine whether the shooting angle of view corresponding to the image to be identified satisfies a preset angle of view range.
[0108] based on Fig.10 In the corresponding example, it can be determined whether the target position is in the second target area (i.e., area 5). If so, it can be determined that the shooting angle corresponding to the image to be identified meets the preset angle range. If not, it is determined that the shooting angle corresponding to the image to be identified does not meet the preset angle range. Fig.10 Since the road vanishing point is in region 8 (not in region 5), it can be determined Fig.10 The shooting angle of view corresponding to the image to be identified does not meet the preset angle of view range.
[0109] Through this method, the images to be identified whose shooting angles are too biased, too upward or too downward can be further filtered out, thereby improving the accuracy of determining available images from the set of images to be identified.
[0110] Next, the artificial intelligence-based image recognition method provided in the embodiment of the present application will be introduced in combination with actual application scenarios.
[0111] In general, the artificial intelligence-based image recognition method provided in the embodiments of the present application is an automated traversal method. Fig.11 , which shows an overall flow chart of an image recognition method based on artificial intelligence provided by an embodiment of the present application, such as Fig.11 As shown, the set of images to be identified collected by crowdsourcing of devices can be input into the intelligent recognition system, and the available images in the set of images to be identified can be identified through the recognition of the intelligent recognition system.
[0112] See also Fig.12 , the figure shows a structural diagram of a cascade intelligent recognition system provided by an embodiment of the present application. After the image set to be recognized is input into the cascade intelligent recognition system, the classification model 1 can be used to identify images in the image set to be recognized that belong to the backlight type, dim type, object occlusion type, reflective occlusion type and tilt type, and these images are determined as unusable images, and images that are uncertain whether they belong to the backlight type, dim type, object occlusion type, reflective occlusion type or tilt type are determined as uncertain images, and the part of images that are determined not to belong to the backlight type, dim type, object occlusion type, reflective occlusion type or tilt type are input into the classification model 2.
[0113] Classification model 2 identifies whether this part of the images belongs to the non-road type and non-sunny type, determines the images that belong to the non-road type and / or non-sunny type as unusable images, determines the images that are uncertain whether they belong to the non-road type and / or non-sunny type as uncertain images, and inputs the images that are determined not to belong to the non-road type and / or non-sunny type into the blur module for blur detection.
[0114] Accordingly, the blur module can perform blur recognition on these images, determine whether there are images that are blurry images, and determine the blurry images as unusable images, determine the images that are uncertain whether they are blurry images as uncertain images, and input the images that are determined not to be blurry images into the shooting angle module for detection. The blur module performs blur recognition in the manner described above and will not be repeated here.
[0115] The shooting angle module can determine whether the image input therein satisfies a preset angle range, determine an image that does not satisfy the preset angle range as an unusable image, determine an image that is uncertain whether it satisfies the preset angle range as an uncertain image, and determine an image that satisfies the preset angle range as an usable image. The recognition method of the shooting angle module is as described above and will not be repeated here.
[0116] In this way, it can be determined efficiently and accurately whether the image to be identified is a usable image.
[0117] Based on the aforementioned artificial intelligence-based image recognition method, the present application embodiment also provides an artificial intelligence-based image recognition device, such as Fig.13 As shown, the figure shows an image recognition device based on artificial intelligence provided by an embodiment of the present application, the device includes an acquisition unit 1301 and a determination unit 1302:
[0118] The acquisition unit 1301 is used to acquire a set of images to be recognized, wherein the set of images to be recognized includes at least two images to be recognized;
[0119] The determining unit 1302 is used to traverse the to-be-recognized images in the to-be-recognized image set, and determine whether the to-be-recognized image belongs to an unusable image type according to the classification model, where the unusable image type includes any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type;
[0120] The determining unit 1302 is further configured to determine, if not, whether the image to be identified is an available image.
[0121] In a possible implementation manner, the determining unit 1302 is further specifically configured to:
[0122] Determining whether the image to be identified is a blurred image;
[0123] If not, it is determined that the image to be identified is an available image.
[0124] In a possible implementation manner, the determining unit 1302 is further specifically configured to:
[0125] intercepting a first target area from the image to be identified, wherein the first target area is a non-sky area;
[0126] generating a grayscale image corresponding to the first target area;
[0127] Convolving the grayscale image according to the convolution kernel to generate a target grayscale image;
[0128] Calculating the grayscale variance of all pixels in the target grayscale image;
[0129] Determine whether the grayscale variance is less than a preset variance threshold, if so, determine that the image to be identified is a blurred image, if not, determine that the image to be identified is not a blurred image.
[0130] In a possible implementation manner, the determining unit 1302 is further specifically configured to:
[0131] Determining whether the shooting angle of view corresponding to the image to be identified meets a preset angle of view range;
[0132] If so, it is determined that the image to be identified is an available image.
[0133] In a possible implementation manner, the determining unit 1302 is further specifically configured to:
[0134] Dividing a second target area corresponding to the preset viewing angle range from the image to be identified;
[0135] Determine a target position of a target object in the image to be identified;
[0136] determining whether the target location is within the second target area;
[0137] If so, it is determined that the shooting angle of view corresponding to the image to be identified satisfies a preset angle of view range.
[0138] In a possible implementation manner, the images to be identified in the set of images to be identified are crowdsourced image data from devices.
[0139] In a possible implementation manner, the unavailable image type further includes one or more combinations of non-road type and non-sunny day type.
[0140] It can be seen from the above technical solution that after obtaining a set of images to be identified including at least two images to be identified, the images to be identified in the set can be automatically traversed, and the classification model can be used to determine whether the image to be identified belongs to an unusable image type, wherein the unusable image type can include any one or more combinations of backlight type, dim type, object occlusion type, reflective occlusion type, and tilt type. If the image to be identified does not belong to an unusable image type, it can be determined that the image to be identified is an available image, i.e., a high-quality image. In this way, it can be efficiently and accurately determined whether the image to be identified is an available image.
[0141] The embodiment of the present application also provides an image recognition device based on artificial intelligence. The image recognition device based on artificial intelligence is introduced below in conjunction with the accompanying drawings. Fig.14 As shown, the embodiment of the present application provides an artificial intelligence-based image recognition device 1400, which can also be a terminal device. The terminal device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a vehicle-mounted computer, etc., and the terminal device is a mobile phone as an example:
[0142] Fig.14 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.14 The mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490. Those skilled in the art can understand that Fig.14 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0143] Combine the following Fig.14 A detailed introduction to the various components of the mobile phone:
[0144] The RF circuit 1410 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1480 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit 1410 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, referred to as LNA), a duplexer, etc. In addition, the RF circuit 1410 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.
[0145] The memory 1420 can be used to store software programs and modules. The processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0146] The input unit 1430 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432. The touch panel 1431, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1431 or near the touch panel 1431 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1431 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1480, and can receive and execute commands sent by the processor 1480. In addition, the touch panel 1431 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1431, the input unit 1430 may also include other input devices 1432. Specifically, other input devices 1432 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.
[0147] The display unit 1440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1440 may include a display panel 1441. Optionally, the display panel 1441 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1431 may cover the display panel 1441. When the touch panel 1431 detects a touch operation on or near it, it is transmitted to the processor 1480 to determine the type of touch event. Subsequently, the processor 1480 provides a corresponding visual output on the display panel 1441 according to the type of touch event. Although in Fig.14 In the embodiment, the touch panel 1431 and the display panel 1441 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1431 and the display panel 1441 can be integrated to realize the input and output functions of the mobile phone.
[0148] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1441 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1441 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.
[0149] The audio circuit 1460, the speaker 1461, and the microphone 1462 can provide an audio interface between the user and the mobile phone. The audio circuit 1460 can transmit the received audio data to the speaker 1461 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1460 and converted into audio data, and then the audio data is output to the processor 1480 for processing, and then sent to another mobile phone through the RF circuit 1410, or the audio data is output to the memory 1420 for further processing.
[0150] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1470. It provides users with wireless broadband Internet access. Fig.14 A WiFi module 1470 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.
[0151] The processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1420, and calling data stored in the memory 1420. Optionally, the processor 1480 may include one or more processing units; preferably, the processor 1480 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1480.
[0152] The mobile phone also includes a power supply 1490 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1480 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.
[0153] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0154] In this embodiment, the processor 1480 included in the terminal device also has the following functions:
[0155] Acquire a set of images to be identified, wherein the set of images to be identified includes at least two images to be identified;
[0156] Traversing the images to be identified in the set of images to be identified, and determining whether the images to be identified belong to an unusable image type according to a classification model, wherein the unusable image type includes any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type;
[0157] If not, it is determined that the image to be identified is an available image.
[0158] The artificial intelligence-based image recognition device provided in the embodiment of the present application may be a server, see Fig.15 As shown, Fig.15The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1522 (for example, one or more processors) and a memory 1532, and one or more storage media 1530 (for example, one or more mass storage devices) storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be short-term storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1522 can be configured to communicate with the storage medium 1530 and execute a series of instruction operations in the storage medium 1530 on the server 1500.
[0159] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0160] The steps performed by the server in the above embodiment can be based on the Fig.15 The server structure shown.
[0161] The CPU 1522 is used to execute the following steps:
[0162] Acquire a set of images to be identified, wherein the set of images to be identified includes at least two images to be identified;
[0163] Traversing the images to be identified in the set of images to be identified, and determining whether the images to be identified belong to an unusable image type according to a classification model, wherein the unusable image type includes any one or more combinations of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, and a tilt type;
[0164] If not, it is determined that the image to be identified is an available image.
[0165] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0166] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0167] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0168] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0169] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0170] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (RandomAccess Memory, referred to as RAM), disk or optical disk and other media that can store program codes.
[0171] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
[0172] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (English: read-only memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store program codes.
[0173] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0174] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An image recognition method based on artificial intelligence, characterized in that: The method is applied to a cascade intelligent recognition system, the system comprising a first classification model, a second classification model, a blur module and a shooting angle module, the cascade intelligent recognition system is used to identify unusable images and uncertain images, and the method comprises: Acquire a set of images to be identified, wherein the set of images to be identified includes at least two images to be identified, and the images to be identified in the set of images to be identified are crowdsourced image data of the device; Traversing the images to be identified in the set of images to be identified, and determining whether the images to be identified belong to unusable image types according to various classification models, wherein the unusable image types include any combination of backlight type, dim type, object occlusion type, reflective occlusion type, tilt type, non-road type, and non-sunny day type; If not, dividing a second target area corresponding to a preset viewing angle range from the image to be identified; Determine a target position of a target object in the image to be identified; determining whether the target location is within the second target area; If so, determining that the shooting angle of view corresponding to the image to be identified meets a preset angle of view range, and determining the image to be identified as an available image; The first classification model is used to identify whether the image to be identified belongs to any one or a combination of the backlight type, dim type, object occlusion type, reflective occlusion type and tilt type; The second classification model is used to identify whether the image to be identified belongs to any one or a combination of non-road type and non-sunny day type; The blur module is used to identify whether the image to be identified is a blurred image; The shooting angle module is used to determine whether the image to be identified meets a preset angle range.
2. The method according to claim 1, characterized in that The determining that the image to be recognized is a usable image includes: Determining whether the image to be identified is a blurred image; If not, it is determined that the image to be identified is an available image.
3. The method according to claim 2, characterized in that The step of determining whether the image to be identified is a blurred image includes: intercepting a first target area from the image to be identified, wherein the first target area is a non-sky area; generating a grayscale image corresponding to the first target area; Convolving the grayscale image according to the convolution kernel to generate a target grayscale image; Calculating the grayscale variance of all pixels in the target grayscale image; Determine whether the grayscale variance is less than a preset variance threshold, if so, determine that the image to be identified is a blurred image, if not, determine that the image to be identified is not a blurred image.
4. An image recognition device based on artificial intelligence, characterized in that: The device is applied to a cascade intelligent recognition system, the system includes a first classification model, a second classification model, a blur module and a shooting angle module, the cascade intelligent recognition system is used to identify unusable images and uncertain images, and the device includes an acquisition unit and a determination unit: The acquisition unit is used to acquire a set of images to be recognized, wherein the set of images to be recognized includes at least two images to be recognized, and the images to be recognized in the set of images to be recognized are crowdsourced image data of the device; The determination unit is used to traverse the images to be identified in the set of images to be identified, and determine whether the images to be identified belong to an unusable image type according to various classification models, wherein the unusable image types include a combination of a backlight type, a dim type, an object occlusion type, a reflective occlusion type, an inclined type, a non-road type, and a non-sunny day type; The determining unit is further configured to determine, if not, that the image to be identified is an available image; The first classification model is used to identify whether the image to be identified belongs to any one or a combination of the backlight type, dim type, object occlusion type, reflective occlusion type and tilt type; The second classification model is used to identify whether the image to be identified belongs to any one or a combination of non-road type and non-sunny day type; The blur module is used to identify whether the image to be identified is a blurred image; The shooting angle module is used to determine whether the image to be identified meets a preset angle range; The determining unit is further specifically configured to: Determining whether the shooting angle of view corresponding to the image to be identified meets a preset angle of view range; If so, determining that the image to be identified is an available image; The determining unit is further specifically configured to: Dividing a second target area corresponding to the preset viewing angle range from the image to be identified; Determine a target position of a target object in the image to be identified; determining whether the target location is within the second target area; If so, it is determined that the shooting angle of view corresponding to the image to be identified satisfies a preset angle of view range.
5. An image recognition device based on artificial intelligence, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the artificial intelligence-based image recognition method described in any one of claims 1-3 according to the instructions in the program code.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the artificial intelligence-based image recognition method described in any one of claims 1-3.
