Image recognition method and device and electronic equipment

Abstract

The invention relates to the technical field of image recognition, in particular to an image recognition method and device and electronic equipment, and is used for solving the problem that the detection efficiency is relatively low when integrity detection is performed on external light. The method comprises the following steps: acquiring a to-be-detected image; performing image segmentation on the to-be-detected image to obtain a contour region image corresponding to the light-emitting device contained in the to-be-detected image; the light-emitting device comprises at least two light-emitting components. Carrying out image classification on the contour region images to obtain classification information of each contour region image; performing discrimination processing on the classification information to obtain an image recognition result of the to-be-detected image; wherein the image recognition result comprises image normality and image abnormality.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular, to an image recognition method, apparatus, and electronic device. Background Art

[0002] Currently, after all assembly procedures of a vehicle are completed in the general assembly workshop, it is necessary to turn on all external lights on the final inspection line for integrity inspection. For example, manually check whether there are defects in the external lights (such as: not lit (overall, partial), inaccurate color, dim, etc.) to ensure the quality of the external lights. However, when using this method to perform integrity inspection on external lights, there is a problem of low inspection efficiency. Summary of the Invention

[0003] To solve the above technical problems, the present disclosure provides an image recognition method, apparatus, and electronic device.

[0004] To achieve the above object, the present disclosure adopts the following technical solutions:

[0005] In a first aspect, the present disclosure provides an image recognition method, including: obtaining an image to be detected; performing image segmentation on the image to be detected to obtain a contour region image corresponding to a light-emitting device included in the image to be detected; wherein the light-emitting device includes at least two light-emitting components; performing image classification on the contour region image to obtain classification information of each contour region image; performing discrimination processing on the classification information to obtain an image recognition result of the image to be detected; wherein the image recognition result includes image normal and image abnormal.

[0006] In some feasible examples, performing image classification on the contour region image to obtain classification information of each contour region image includes: performing image screening based on the image information of the contour region image to obtain a target image; wherein the image information includes the total number of pixel points or the center point position, the center point position includes the actual center point position and the theoretical center point position, and the target image includes any one of the contour region images; performing image classification on the target image to obtain classification information of each target image.

[0007] In some feasible examples, the image information includes the total number of pixel points; performing image screening on the contour region image to obtain a target image includes: performing image screening based on the total number of pixel points included in each contour region image, and taking the contour region image with the total number of pixel points greater than a prior threshold as the target image.

[0008] In some feasible examples, the image information includes the actual center point position and the theoretical center point position; screening the contour region images to obtain target images, including: calculating the distance based on the actual center point position and the theoretical center point position corresponding to each contour region image to obtain the actual distance between the actual center point position and the theoretical center point position; screening the images based on the actual distance, and taking the contour region images with the actual distance less than the distance threshold as target images.

[0009] In some feasible examples, classifying the contour region images to obtain the classification information of each contour region image, including: extracting features from the contour region images to obtain the feature maps corresponding to the contour region images; converting the storage format of the feature maps from two-dimensional to one-dimensional to obtain a one-dimensional vector with a length of w×h; where w is the width of the contour region image and h is the length of the contour region image; multiplying each element in the one-dimensional vector by the abnormal weight and the normal weight respectively to obtain the abnormal value and the normal value corresponding to each element; summing the abnormal values corresponding to each element to obtain the total abnormal value in the classification information; summing the normal values corresponding to each element to obtain the total normal value in the classification information.

[0010] In some feasible examples, performing discriminant processing on the classification information to obtain the image recognition result of the image to be detected, including: in the case where there is classification information with the total abnormal value greater than or equal to the total normal value, determining that the image recognition result of the image to be detected is image abnormal.

[0011] In a second aspect, the present disclosure provides an image recognition device, including: an acquisition unit for acquiring an image to be detected; a processing unit for performing image segmentation on the image to be detected acquired by the acquisition unit to obtain the contour region images corresponding to the light-emitting devices included in the image to be detected; where the light-emitting device includes at least two light-emitting components; the processing unit is further configured to classify the contour region images to obtain the classification information of each contour region image; the processing unit is further configured to perform discriminant processing on the classification information to obtain the image recognition result of the image to be detected; where the image recognition result includes image normal and image abnormal.

[0012] In some feasible examples, the processing unit is specifically configured to screen the images based on the image information of the contour region images to obtain target images; where the image information includes the total number of pixel points or the center point position, the center point position includes the actual center point position and the theoretical center point position, and the target images include any one of the contour region images; the processing unit is specifically configured to classify the target images to obtain the classification information of each target image.

[0013] In some implementable examples, the processing unit is specifically configured to perform image screening based on the total number of pixel points included in each contour region image, and use the contour region image with the total number of pixel points greater than the prior threshold as the target image.

[0014] In some implementable examples, the image information includes the actual center point position and the theoretical center point position; the processing unit is specifically configured to calculate the distance based on the actual center point position and the theoretical center point position corresponding to each contour region image to obtain the actual distance between the actual center point position and the theoretical center point position; the processing unit is specifically configured to perform image screening based on the actual distance, and use the contour region image with the actual distance less than the distance threshold as the target image.

[0015] In some implementable examples, the processing unit is specifically configured to extract features from the contour region image to obtain the feature map corresponding to the contour region image; the processing unit is specifically configured to convert the storage format of the feature map from two-dimensional to one-dimensional to obtain a one-dimensional vector with a length of w×h; where w is the width of the contour region image and h is the length of the contour region image; the processing unit is specifically configured to multiply each element in the one-dimensional vector by the abnormal weight and the normal weight respectively to obtain the abnormal value and the normal value corresponding to each element; the processing unit is specifically configured to sum the abnormal values corresponding to each element to obtain the total abnormal value in the classification information; the processing unit is specifically configured to sum the normal values corresponding to each element to obtain the total normal value in the classification information.

[0016] In some implementable examples, the processing unit is specifically configured to determine that the image recognition result of the image to be detected is image abnormality in the case where there is classification information with the total abnormal value greater than or equal to the total normal value.

[0017] In a third aspect, the present disclosure provides an electronic device, including: a memory and a processor, the memory is used to store a computer program; the processor is used to cause the electronic device to implement the image recognition method provided in the first aspect as described above when executing the computer program.

[0018] In a fourth aspect, the present disclosure provides a computer-readable storage medium, including: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, it causes the computing device to implement the image recognition method provided in the first aspect as described above.

[0019] In a fifth aspect, the present disclosure provides a vehicle, including any one of the image recognition devices provided in the second aspect.

[0020] In the present disclosure, the names of the above-mentioned image recognition devices do not constitute limitations on the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of the present disclosure and fall within the scope of the claims of the present disclosure and their equivalent technologies.

[0021] These aspects or other aspects of the present disclosure will be more clearly understood in the following description.

[0022] The technical solutions provided by the present disclosure have the following advantages compared with the prior art:

[0023] By performing image segmentation on the image to be detected, the contour region image corresponding to the light-emitting device included in the image to be detected can be determined. Then, by performing image classification on each contour region image, the classification information of each contour region image is obtained; in this way, the image recognition result of the image to be detected can be obtained according to the classification information. It can be seen that when it is necessary to detect external lights (such as light-emitting devices), only the light image to be recognized corresponding to the external light needs to be acquired, and then the external light can be detected, without the need for manual inspection of whether there are defects in the external light. Therefore, more vehicles' external lights can be inspected for defects per unit time, improving the detection efficiency and solving the problem of low detection efficiency in the prior art when performing integrity detection on external lights. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is one of the schematic flowcharts of an image recognition method provided by an embodiment of the present disclosure;

[0027] Figure 2 It is a schematic diagram of the light image to be recognized in an image recognition method provided by an embodiment of the present disclosure;

[0028] Figure 3 It is a schematic diagram of the image corresponding to the ROI in an image recognition method provided by an embodiment of the present disclosure;

[0029] Figure 4 It is the second schematic flowchart of an image recognition method provided by an embodiment of the present disclosure;

[0030] Figure 5 The third flowchart of an image recognition method provided by an embodiment of the present disclosure;

[0031] Figure 6 The fourth flowchart of an image recognition method provided by an embodiment of the present disclosure;

[0032] Figure 7 The fifth flowchart of an image recognition method provided by an embodiment of the present disclosure;

[0033] Figure 8 The sixth flowchart of an image recognition method provided by an embodiment of the present disclosure;

[0034] Figure 9 The structural schematic diagram of an image recognition device provided by an embodiment of the present disclosure;

[0035] Figure 10 The structural schematic diagram of an image recognition server provided by an embodiment of the present disclosure;

[0036] Figure 11 The structural schematic diagram of a computer program product of an image recognition method provided by an embodiment of the present disclosure. Detailed implementation manners

[0037] In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0038] Many specific details are set forth in the following description in order to fully understand the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0039] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0040] The external lights in the embodiments of the present disclosure refer to the lights visible outside the vehicle, such as brake lights and turn signals, and the lights generated when they are turned on.

[0041] ROI in the embodiments of the present disclosure refers to Region Of Interest, which is the light-emitting area corresponding to the light-emitting device in the image.

[0042] The execution subject of the image recognition method in the embodiments of the present application is an image recognition device, which can be set in a server or other devices, and this embodiment does not limit this.

[0043] The Mask rcnn algorithm in the embodiments of the present application is composed of the faster rcnn and the semantic segmentation algorithm FCN.

[0044] The SSD series in the embodiments of the present application belongs to the one_stage regression-based object detection method, including: SSD (Single Shot MultiBox Detector) algorithm, DSSD (Deconvolutional Single ShotDetector) algorithm, FSSD (Feature Fusion Single Shot Multibox Detector) algorithm, RefineDet (Single-Shot Refinement Neural Network for Object Detection) algorithm, RfbNet (Receptive Field Block Net for Accurate and Fast Object Detection) algorithm, M2Det (A Single-Shot Object Detector based on Multi-Level Feature PyramidNetwork) algorithm, etc.

[0045] The YOLO (You Only Look Once) series in the embodiments of the present application is a one-stage and deep learning-based regression method, including YOLOv1, YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv6, and YOLOv7, etc.

[0046] Exemplarily, taking the server as the execution subject for executing the image recognition method provided in the embodiments of the present disclosure as an example, the image recognition method provided in the embodiments of the present disclosure will be introduced.

[0047] Such as Figure 1As shown in the figure, the image recognition method provided by the embodiments of the present disclosure includes the following steps S11 - S14:

[0048] S11. Obtain the image to be detected.

[0049] In some examples, one or more lighting devices (such as brake lights, turn signals, etc.) are installed on a vehicle. To ensure that the lights installed on the vehicle can work properly, the lights need to be inspected. For example, after all assembly procedures of the vehicle are completed in the general assembly workshop, all lighting devices on the vehicle are turned on and photographed on the final inspection line, so as to obtain the image to be detected. Then, by processing the image to be detected, the image recognition result of the image to be detected is determined.

[0050] S12. Perform image segmentation on the image to be detected to obtain a contour region image corresponding to the lighting device included in the image to be detected. Among them, the lighting device includes at least two lighting components.

[0051] In some examples, the image to be detected can be input into an entity segmenter for image segmentation to obtain a contour region image corresponding to the lighting device included in the image to be detected. Among them, the training process of the entity segmenter is as follows:

[0052] Obtain the first training sample data and the first labeling result of the first training sample data. Among them, the first training sample data includes a certain number (such as 100) of historical detection images of the lighting device of the vehicle when it is lit; the first labeling result includes: the ROI corresponding to the lighting device included in the historical detection image, and the number of ROIs (such as: the number a ∈ [0, 10], a is an integer greater than or equal to 0).

[0053] Input the first training sample data into a deep learning instance segmentation network for learning to obtain the first prediction result of the deep learning instance segmentation network for the first training sample data. Among them, the deep learning instance segmentation network includes the MaskRCNN algorithm.

[0054] In the case where the first prediction result is different from the first labeling result, use a target algorithm (such as the Error BackPropagation Algorithm (BP) algorithm, or the Radial Basis Function (RBF) algorithm) to adjust the network parameters of the deep learning instance segmentation network until the convergence condition is met, and save the network parameters of the deep learning instance segmentation network when the first convergence condition is met to obtain the entity segmenter.

[0055] Specifically, the first convergence condition includes that the ratio of the number of times the first prediction result is the same as the first labeling result to the number of times the first prediction result is different from the first labeling result is greater than or equal to a preset threshold. Alternatively, the number of loop iterations of the deep learning instance segmentation network is equal to the target threshold (e.g., 200 times).

[0056] Exemplarily, when the vehicle's lamp is lit, it displays the to-be-detected image 1 as shown below. It can be seen that the to-be-detected image 1 contains 3 ROIs, namely the light area 1-1, the light area 1-2, and the light area 1-3. Figure 2 As shown.

[0057] Exemplarily, the contour area image corresponding to the ROI is as shown below. Figure 3 As shown.

[0058] S13. Perform image classification on the contour area image to obtain the classification information of each contour area image.

[0059] In some examples, the contour area image can be input into a light defect recognizer for image classification to obtain the classification information of each contour area image. Among them, the training process of the light defect recognizer is as follows:

[0060] Obtain the second training sample data and the labeling result of the second training sample data. Among them, the second training sample data includes at least one contour area image corresponding to the ROI, and the labeling result includes the labeling type of the contour area image corresponding to the ROI. The labeling type includes any one of image normal and image abnormal.

[0061] Specifically, when obtaining the second training sample data, the minimum bounding rectangle of the contour area image corresponding to each ROI can be obtained, and the background content outside the minimum bounding rectangle can be set to zero. Then, the image corresponding to the processed minimum bounding rectangle can be uniformly scaled to a resolution of W*H to obtain the second training sample data. Among them, typical values of W and H include but are not limited to W = 224, H = 224 or W = 128, H = 128.

[0062] Input the second training sample data into a classification task network for learning to obtain the second prediction result of the second training sample data for the training sample data. Among them, the classification task network includes any one of the Residual Network (ResNet) series, the MobileNet series, the ShuffleNet series, the SENet series, and the Transformer series.

[0063] In the case where the second prediction result is different from the second labeled result, the network parameters of the classification task network are adjusted using the target algorithm until the convergence condition is met, and the network parameters of the classification task network when the second convergence condition is satisfied are saved to obtain a lighting defect identifier.

[0064] Specifically, the second convergence condition includes that the ratio of the number of times the second prediction result is the same as the second labeled result to the number of times the second prediction result is different from the second labeled result is greater than or equal to a preset threshold. Or, the number of loop iterations of the classification task network is equal to a target threshold (such as 200 times).

[0065] In some examples, in order to improve the classification efficiency of image classification, it is necessary to screen the contour region images. For example, screen the contour region images based on the total number of pixel points included in the contour region images; or screen the contour region images based on the actual center point coordinates and the theoretical center point coordinates of the contour region images; or screen the contour region images based on the total number of pixel points included in the contour region images, as well as the actual center point coordinates and the theoretical center point coordinates of the contour region images. In this way, the number of contour region images can be greatly reduced, thereby improving the classification efficiency of image classification.

[0066] S14. Perform discriminant processing on the classification information to obtain the image recognition result of the image to be detected; where the image recognition result includes normal image and abnormal image.

[0067] As can be seen from the above, when external lighting detection is required, the server only needs to obtain the image to be detected corresponding to the external lighting, and then can perform external lighting detection, without the need for manual inspection of whether there are defects in the external lighting. Therefore, more vehicles' external lighting can be inspected for defects per unit time, improving the detection efficiency and solving the problem of low detection efficiency in the prior art when performing integrity detection on external lighting.

[0068] In some implementable examples, combined Figure 1 , such as Figure 4 shown, the above S13 can be specifically implemented through the following S130 and S131.

[0069] S130. Perform image screening based on the image information of the contour region image to obtain a target image. Where the image information includes the total number of pixel points or the center point position, and the center point position includes the actual center point position and the theoretical center point position, and the target image includes any one of the contour region images.

[0070] In some examples, an image acquisition device (such as a camera) can capture a vehicle parked at a target position in different shooting postures at a specified position, so as to obtain an image to be detected. In this way, for vehicles of the same model, an initial image can be captured at each shooting position in the same shooting posture, and the central point coordinates of the contour area image in the initial image can be used as the theoretical central point coordinates. In this way, in the subsequent process, based on the shooting position and shooting posture of the image to be detected, and the model of the current vehicle, the theoretical central point coordinates can be obtained.

[0071] In some examples, based on the actual distance between the actual central point coordinates and the theoretical central point coordinates, the actual distance. Alternatively, a circle is drawn with the actual central point coordinates as the center according to a preset radius to obtain a first area; at the same time, a circle is drawn with the theoretical central point coordinates as the center according to a preset radius to obtain a second area; then, the total number of pixel points that belong to both the first area and the second area is determined. When the ratio of the total number of pixel points to the total number of pixel points included in the first area is greater than a specified threshold, and the ratio of the total number of pixel points to the total number of pixel points included in the second area is greater than the specified threshold, the image corresponding to the actual central point coordinates is used as the filtered target image.

[0072] S131. Perform image classification on the target image to obtain the classification information of each target image.

[0073] In some implementable examples, the image information includes the total number of pixel points; combined Figure 4 , such as Figure 5 shown, the above S130 can be specifically implemented by the following S1300.

[0074] S1300. Perform image screening based on the total number of pixel points included in each contour area image, and use the contour area image with the total number of pixel points greater than the prior threshold as the target image.

[0075] In some examples, the contour area image can be segmented into pixel points, and then the pixel points after segmentation can be counted to obtain the total number of pixel points included in the contour area image.

[0076] In some examples, the contour area image can be input into a pixel point recognition model to obtain the total number of pixel points included in the contour area image. Among them, the training process of the pixel point recognition model is as follows:

[0077] Obtain training sample data and the labeling results of the training sample data; among them, the training sample data includes at least one contour area image, and the labeling results include the total number of pixel points included in the contour area image.

[0078] Input the training sample data into the neural network model for learning to obtain the prediction result of the neural network model for the training sample data.

[0079] Adjust the network parameters of the neural network model based on the prediction result and the labeled result until the neural network model converges to obtain the pixel point recognition model.

[0080] In some examples, the prior threshold includes any one of 10, 20, and 30.

[0081] Exemplarily, assume the prior threshold is equal to 10. When the total number of pixel points is greater than or equal to 10, the contour region image corresponding to the total number of pixel points is used as the filtered target image. When the total number of pixel points is less than 10, the contour region image corresponding to the total number of pixel points is excluded.

[0082] In some feasible examples, the image information includes the actual center point position and the theoretical center point position; combined Figure 4 , as Figure 6 shown, the above S130 can be specifically implemented by the following S1301 and S1302.

[0083] S1301. Calculate the distance based on the actual center point position and the theoretical center point position corresponding to each contour region image to obtain the actual distance between the actual center point position and the theoretical center point position.

[0084] Exemplarily, taking the actual center point coordinates and the theoretical center point coordinates as the coordinate points in the two-dimensional rectangular coordinate system as an example, the process of calculating the actual distance between the actual center point coordinates and the theoretical center point coordinates is as follows:

[0085] Substitute the actual center point coordinates and the theoretical center point coordinates into the distance calculation formula to obtain the difference between the actual center point coordinates and the theoretical center point coordinates. Among them, the distance calculation formula includes:

[0086]

[0087] Among them, d represents the difference between the actual center point coordinates and the theoretical center point coordinates, x1 represents the abscissa of the actual center point coordinates, y1 represents the ordinate of the actual center point coordinates, x2 represents the abscissa of the theoretical center point coordinates, and y2 represents the ordinate of the theoretical center point coordinates.

[0088] After that, take the difference between the actual center point coordinates and the theoretical center point coordinates as the actual distance between the actual center point coordinates and the theoretical center point coordinates.

[0089] S1302. Perform image screening based on the actual distance, and use the contour region image with the actual distance less than the distance threshold as the target image.

[0090] In some examples, the distance threshold includes any one of 0.1, 0.2, 0.3, and 0.5.

[0091] Exemplarily, assuming that the distance threshold is equal to 0.1, if the actual distance is less than 0.1, the contour region image corresponding to the actual distance is used as the filtered target image. If the actual distance is greater than or equal to 0.1, the contour region image corresponding to the actual distance is excluded.

[0092] In some examples, when the actual distance in the image is greater than the distance threshold, it indicates that the two are not images of the same lamp. Therefore, the contour region image with an actual distance greater than the distance threshold can be excluded to improve the classification efficiency of image classification.

[0093] In some implementable examples, in combination with Figure 1 , such as Figure 7 shown, the above S13 can be specifically implemented through the following S132 - S136.

[0094] S132. Extract features from the contour region image to obtain the feature map corresponding to the contour region image.

[0095] S133. Convert the storage format of the feature map from two - dimensional to one - dimensional to obtain a one - dimensional vector of length w×h; where w is the width of the contour region image and h is the length of the contour region image.

[0096] S134. Multiply each element in the one - dimensional vector by the abnormal weight and the normal weight respectively to obtain the abnormal value and the normal value corresponding to each element.

[0097] S135. Sum up the abnormal values corresponding to each element to obtain the total abnormal value in the classification information.

[0098] S136. Sum up the normal values corresponding to each element to obtain the total normal value in the classification information.

[0099] In some implementable examples, in combination with Figure 1 , such as Figure 8 shown, the above S14 can be specifically implemented through the following S140.

[0100] S140. In the case where there is classification information with the total abnormal value greater than or equal to the total normal value, determine that the image recognition result of the image to be detected is image abnormal.

[0101] In some examples, in the case where there is classification information with an abnormal total value greater than or equal to the normal total value, the image recognition result of the image to be detected is determined to be an abnormal image. In the case where there is no classification information with an abnormal total value greater than or equal to the normal total value, the image recognition result of the image to be detected is determined to be a normal image. It can be seen that through the image recognition method provided in the embodiments of the present disclosure, when external light detection is required, only the light image to be recognized corresponding to the external light needs to be acquired, and then the external light detection can be performed, without the need for manual inspection of whether there are defects in the external light. Therefore, the defects in the external lights of more vehicles can be inspected per unit time, improving the detection efficiency.

[0102] In some feasible examples, the resolution of the image to be detected is a preset resolution; wherein, the preset resolution includes any one of a first resolution and a second resolution. The height of the first resolution is 224, and the width is 224. The height of the second resolution is 128, and the width is 128.

[0103] The above mainly introduces the solution provided by the embodiments of the present disclosure from the perspective of the method. To implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0104] The embodiments of the present disclosure can respectively divide the function modules of the image recognition device according to the above method examples. For example, each function module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module. It should be noted that the division of modules in the embodiments of the present disclosure is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0105] As Figure 9 shown, it is a schematic structural diagram of an image recognition device 10 provided by the embodiments of the present disclosure. It includes an acquisition unit 101 and a processing unit 102.

[0106] An acquisition unit 101 is configured to acquire an image to be detected; a processing unit 102 is configured to perform image segmentation on the image to be detected acquired by the acquisition unit 101 to obtain a contour region image corresponding to a light-emitting device included in the image to be detected; wherein the light-emitting device includes at least two light-emitting components; the processing unit 102 is further configured to perform image classification on the contour region image to obtain classification information of each contour region image; the processing unit 102 is further configured to perform discrimination processing on the classification information to obtain an image recognition result of the image to be detected; wherein the image recognition result includes image normal and image abnormal.

[0107] In some feasible examples, the processing unit 102 is specifically configured to perform image screening based on the image information of the contour region image to obtain a target image; wherein the image information includes the total number of pixel points or the center point position, and the center point position includes the actual center point position and the theoretical center point position, and the target image includes any one of the contour region images; the processing unit 102 is specifically configured to perform image classification on the target image to obtain classification information of each target image.

[0108] In some feasible examples, the processing unit 102 is specifically configured to perform image screening based on the total number of pixel points included in each contour region image, and use the contour region image with the total number of pixel points greater than the prior threshold as the target image.

[0109] In some feasible examples, the image information includes the actual center point position and the theoretical center point position; the processing unit 102 is specifically configured to calculate the actual distance between the actual center point position and the theoretical center point position based on the actual center point position and the theoretical center point position corresponding to each contour region image; the processing unit 102 is specifically configured to perform image screening based on the actual distance, and use the contour region image with the actual distance less than the distance threshold as the target image.

[0110] In some feasible examples, the processing unit 102 is specifically configured to extract features from the contour region image to obtain a feature map corresponding to the contour region image; the processing unit 102 is specifically configured to convert the storage format of the feature map from two-dimensional to one-dimensional to obtain a one-dimensional vector with a length of w×h; wherein w is the width of the contour region image and h is the length of the contour region image; the processing unit 102 is specifically configured to multiply each element in the one-dimensional vector by an abnormal weight and a normal weight respectively to obtain an abnormal value and a normal value corresponding to each element; the processing unit 102 is specifically configured to sum up the abnormal values corresponding to each element to obtain the total abnormal value in the classification information; the processing unit 102 is specifically configured to sum up the normal values corresponding to each element to obtain the total normal value in the classification information.

[0111] In some feasible examples, the processing unit 102 is specifically configured to determine that the image recognition result of the image to be detected is an image abnormality when there is classification information that the total abnormal value is greater than or equal to the total normal value.

[0112] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0113] Of course, the image recognition device 10 provided in the embodiment of the present disclosure includes but is not limited to the above modules, for example, the image recognition device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the image recognition device 10, and may also be used to store data generated by the image recognition device 10 during operation, such as data in a write request, etc.

[0114] Figure 10 A schematic diagram of the structure of an image recognition server provided in an embodiment of the present disclosure is shown in FIG. Figure 10 As shown, the image recognition server may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .

[0115] Combine the following Figure 10 The following is a detailed introduction to the components of the image recognition server:

[0116] The processor 51 is the control center of the electronic device 10, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0117] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Figure 10 Also, as an embodiment, the electronic device may include multiple processors, such as Figure 10 51 and 55 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0118] The memory 52 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 can exist independently and be connected to the processor 51 through the communication bus 54. The memory 52 can also be integrated with the processor 51.

[0119] In a specific implementation, the memory 52 is used to store the data in the present disclosure and execute the software programs in the present disclosure. The processor 51 can execute various functions of the air conditioner by running or executing the software programs stored in the memory 52 and calling the data stored in the memory 52.

[0120] The communication interface 53 uses any device such as a transceiver to communicate with other devices or communication networks, such as a radio access network (RAN), a wireless local area network (WLAN), a terminal, the cloud, etc. The communication interface 53 can include an acquisition unit 101 to implement the acquisition function.

[0121] The communication bus 54 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0122] As an example, in combination with Figure 9 , the function implemented by the acquisition unit 101 in the image recognition device 10 is the same as that of Figure 10 's communication interface 53, and the function implemented by the processing unit 102 in the image recognition device 10 is the same as that of Figure 10 's processor 51, and the function implemented by the storage unit 103 in the image recognition device 10 is the same as that of Figure 10 's memory 52.

[0123] Another embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a computing device, the computing device is caused to execute the mode switching method shown in the above method embodiment.

[0124] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a computer-readable storage medium in a machine-readable format or encoded in other non-transitory media or articles.

[0125] Figure 11 Schematically shows a conceptual partial view of a computer program product provided by an embodiment of the present disclosure. The computer program product includes a computer program for executing a computer process on a computing device.

[0126] In one embodiment, the computer program product is provided using a signal-bearing medium 410. The signal-bearing medium 410 may include one or more program instructions that, when run by one or more processors, may provide the functions or partial functions described above for Figure 1 . Thus, for example, referring to the embodiment shown in Figure 1 , one or more features of S11 - S14 may be borne by one or more instructions associated with the signal-bearing medium 410. In addition, Figure 11 's program instructions also describe example instructions.

[0127] In some examples, the signal-bearing medium 410 may include a computer-readable medium 411, such as but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and so on.

[0128] In some embodiments, the signal-bearing medium 410 may include a computer-recordable medium 412, such as but not limited to, a memory, a read / write (R / W) CD, an R / W DVD, and so on.

[0129] In some embodiments, the signal-bearing medium 410 may include a communication medium 413, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0130] The signal-bearing medium 410 may be conveyed by a wireless form of the communication medium 413 (e.g., a wireless communication medium compliant with the IEEE 802.41 standard or other transmission protocols). One or more program instructions may be, for example, computing device-executable instructions or logic-implemented instructions.

[0131] In some examples, such as for Figure 9 the image recognition device 10 described, may be configured to provide various operations, functions, or actions in response to one or more program instructions via the computer-readable medium 411, the computer-recordable medium 412, and / or the communication medium 413.

[0132] From the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0133] In several embodiments provided by the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated. The components shown as units may be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, in each embodiment of the present disclosure, each functional unit may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0136] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which may be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0137] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image recognition method, characterized in that, Including: Obtain the image to be detected; Perform image segmentation on the image to be detected to obtain a contour region image corresponding to the light-emitting device included in the image to be detected; wherein, the light-emitting device includes at least two light-emitting components; Perform image classification on the contour region image to obtain classification information for each of the contour region images; Perform discriminant processing on the classification information to obtain an image recognition result of the image to be detected; wherein, the image recognition result includes normal image and abnormal image.

2. The image recognition method according to claim 1, wherein, The performing image classification on the contour region image to obtain classification information for each of the contour region images includes: Perform image screening based on the image information of the contour region image to obtain a target image; wherein, the image information includes the total number of pixel points or the center point position, the center point position includes the actual center point position and the theoretical center point position, and the target image includes any one of the contour region images; Perform image classification on the target image to obtain classification information for each of the target images.

3. The image recognition method according to claim 2, wherein The image information includes the total number of pixel points; the performing image screening on the contour region image to obtain a target image includes: Perform image screening based on the total number of pixel points included in each of the contour region images, and use the contour region image with the total number of pixel points greater than the prior threshold as the target image.

4. The image recognition method according to claim 2, wherein The image information includes the actual center point position and the theoretical center point position; The performing image screening on the contour region image to obtain a target image includes: Calculate the distance based on the actual center point position and the theoretical center point position corresponding to each of the contour region images to obtain the actual distance between the actual center point position and the theoretical center point position; Perform image screening based on the actual distance, and use the contour region image with the actual distance less than the distance threshold as the target image.

5. The image recognition method according to claim 1, wherein, The performing image classification on the contour region image to obtain classification information for each of the contour region images includes: Extract features from the contour region image to obtain a feature map corresponding to the contour region image; Convert the storage format of the feature map from two-dimensional to one-dimensional to obtain a one-dimensional vector with a length of w×h; wherein, w is the width of the contour region image and h is the length of the contour region image; Multiply each element in the one-dimensional vector by an abnormal weight and a normal weight respectively to obtain an abnormal value and a normal value corresponding to each element; Sum the abnormal values corresponding to each element to obtain the total abnormal value in the classification information; Sum the normal values corresponding to each element to obtain the total normal value in the classification information.

6. The image recognition method according to claim 1, wherein The performing discriminant processing on the classification information to obtain the image recognition result of the image to be detected includes: In the case where there is classification information with the total abnormal value greater than or equal to the total normal value, determine that the image recognition result of the image to be detected is an abnormal image.

7. An image recognition device, characterized in that, Including: An acquisition unit for acquiring the image to be detected; A processing unit for performing image segmentation on the to-be-detected image acquired by the acquisition unit to obtain a contour region image corresponding to the light-emitting device included in the to-be-detected image; wherein, the light-emitting device includes at least two light-emitting components; The processing unit is further configured to perform image classification on the contour region image to obtain classification information of each contour region image; The processing unit is further configured to perform discrimination processing on the classification information to obtain an image recognition result of the to-be-detected image; wherein, the image recognition result includes image normal and image abnormal.

8. An electronic device, characterized in that, Comprising: A memory and a processor, the memory is used for storing a computer program; the processor is used for enabling the electronic device to implement the image recognition method according to any one of claims 1-6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, Comprising: A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device is enabled to implement the image recognition method according to any one of claims 1-6.

10. An image recognition server, comprising the image recognition device according to claim 7.

11. An image recognition system, characterized in that, Comprising the image recognition server according to claim 10.

Images