Image processing method, device and system, computer equipment and storage medium

Abstract

The invention relates to an image processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a target detection image of a target object in a to-be-detected image; the target detection image is obtained by processing the to-be-detected image according to a target detection result of the target object; and through a trained integrity evaluation model, outputting an integrity evaluation result for the target object in the target detection image, wherein the integrity evaluation model is obtained by training a sample image obtained based on cutting processing and an integrity label corresponding to the sample image; the integrity label is obtained based on the cutting proportion of the sample image in the cutting processing process. By adopting the method, the evaluation accuracy of the integrity of the target object can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, system, computer equipment, storage medium and computer program product. Background Art

[0002] With the continuous development of image processing and artificial intelligence technology, image detection technology has been widely used in security monitoring, traffic management and other fields.

[0003] In actual scenarios, monitoring equipment can capture multiple images for image detection. For example, in traffic management scenarios, when a vehicle passes through a target road section, the camera will capture multiple vehicle images. If all images are used for subsequent image processing technology, it will be time-consuming and laborious, and some images contain incomplete target objects (such as vehicles), wasting a lot of computer computing resources. Therefore, how to determine the integrity of the target object in the captured image has become an urgent problem to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method, apparatus, system, computer device, computer-readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides an image processing method. The method comprises:

[0006] Acquire a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object;

[0007] Outputting, by means of a trained integrity assessment model, an integrity assessment result for the target object in the target detection image;

[0008] The integrity assessment model is obtained by training a sample image obtained through shearing processing and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing processing.

[0009] In one of the embodiments, before outputting the integrity evaluation result for the target object in the target detection image through the trained integrity evaluation model, the method further includes:

[0010] Performing image shearing processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image;

[0011] Inputting the sample image into an initial completeness assessment model to be trained to obtain a predicted completeness of the sample image;

[0012] Obtaining a loss value of the initial integrity assessment model according to the predicted integrity and the integrity label corresponding to the sample image;

[0013] Based on the loss value, the initial integrity assessment model is iteratively updated to obtain the integrity assessment model.

[0014] In one embodiment, performing image shearing processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image includes:

[0015] Determining the shearing ratio of the initial sample image;

[0016] According to the shearing ratio, obtaining a shearing size of the initial sample image;

[0017] The width and height of the initial sample image are sheared according to the shearing size to obtain the sample image, and the shearing ratio is set as the integrity label corresponding to the sample image.

[0018] In one embodiment, obtaining the cropping size corresponding to the initial sample image according to the cropping ratio includes:

[0019] According to the shearing ratio, obtaining target width information and target height information of the initial sample image;

[0020] According to the target width information, obtaining a first cutting start point and a first cutting end point of the initial sample image at the width;

[0021] According to the target height information, obtain a second shearing starting point and a second shearing end point of the initial sample image at the height;

[0022] The shearing size is obtained according to the first shearing starting point, the first shearing end point, the second shearing starting point and the second shearing end point.

[0023] In one embodiment, obtaining the loss value of the initial integrity assessment model according to the predicted integrity and the integrity label includes:

[0024] Determining probability distribution information and classification information corresponding to the prediction completeness;

[0025] A loss value of the initial integrity assessment model is obtained according to the probability distribution information and the classification information.

[0026] In one embodiment, obtaining a target detection image of a target object in an image to be detected includes:

[0027] Performing target detection processing on the image to be detected by using the trained target detection model to obtain a target detection result for the target object;

[0028] According to the target detection result, image shearing processing is performed on the image to be detected to obtain the target detection image.

[0029] In one embodiment, after obtaining the integrity evaluation result of the target detection image, the method further includes:

[0030] Based on the completeness evaluation result, determining a target processing image that meets a preset completeness condition from the target detection images of the plurality of images to be detected;

[0031] Perform recognition processing on the target processing image.

[0032] In a second aspect, the present application also provides an image processing device. The device comprises:

[0033] An image detection module is used to obtain a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object;

[0034] An image evaluation module is used to output an integrity evaluation result for the target object in the target detection image through a trained integrity evaluation model; wherein the integrity evaluation model is trained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

[0035] In a third aspect, the present application also provides an image processing system. The system comprises:

[0036] The shooting device is used to collect the image to be detected of the target scene and send it to the server;

[0037] The server is used to obtain a target detection image of the target object in the received image to be detected; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object; and outputting the integrity assessment result for the target object in the target detection image through a trained integrity assessment model; wherein the integrity assessment model is obtained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

[0038] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0039] Acquire a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object;

[0040] Outputting, by means of a trained integrity assessment model, an integrity assessment result for the target object in the target detection image;

[0041] The integrity assessment model is obtained by training a sample image obtained through shearing processing and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing processing.

[0042] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object;

[0044] Outputting, by means of a trained integrity assessment model, an integrity assessment result for the target object in the target detection image;

[0045] The integrity assessment model is obtained by training a sample image obtained through shearing processing and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing processing.

[0046] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0047] Acquire a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object;

[0048] Outputting, by means of a trained integrity assessment model, an integrity assessment result for the target object in the target detection image;

[0049] The integrity assessment model is obtained by training a sample image obtained through shearing processing and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing processing.

[0050] The above-mentioned image processing method, device, system, computer equipment, storage medium and computer program product obtain a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object; and the integrity evaluation result for the target object in the target detection image is output through a trained integrity evaluation model; wherein the integrity evaluation model is obtained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing process. By adopting this method, the integrity evaluation model obtained by training sample images of different integrity levels is used to realize the judgment of the integrity of the target object in the target detection image, and the accuracy of the evaluation of the integrity of the target object is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0052] Figure 2 is a schematic flow chart of an image processing method in one embodiment;

[0053] Figure 3 A schematic flow chart of the steps of obtaining a sample image and a completeness label corresponding to the sample image in one embodiment;

[0054] Figure 4 is a schematic flow chart of an image processing method in another embodiment;

[0055] Figure 5 A schematic diagram of a process for obtaining a cut vehicle image in one embodiment;

[0056] Figure 6 A schematic diagram of a process flow of a training process of a completeness assessment model in one embodiment;

[0057] Figure 7 is a schematic diagram of an image to be detected and a target detection image in one embodiment;

[0058] Figure 8 is a structural block diagram of an image processing device in one embodiment;

[0059] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data and image data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0062] The image processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the application environment can be a smart transportation system or a smart home system, and the application environment can include: a router, a gateway, a terminal (such as a mobile phone, a tablet computer, a laptop), a shooting device 101 (such as a camera), a server 102, etc. In the smart transportation system or the smart home system, the shooting device 101 can be connected to the gateway via ZIGBEE / Bluetooth / WiFi, and the gateway and the terminal can be connected to the router via WiFi respectively. In addition, the terminal can also establish a network connection with the server through 2G / 3G / 4G / 5G, WiFi, etc., so as to obtain the data sent by the server.

[0063] The terminal is used to provide local services for users. The server can be implemented by an independent server or a server cluster composed of multiple servers, or by a cloud server.

[0064] In one embodiment, Figure 2 As shown, an image processing method is provided, which can be applied to Figure 1 The gateway, terminal, photographing device 101 or server shown in the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0065] Step S201 , obtaining a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object.

[0066] The target object refers to an object that needs to be detected or identified; for example, the target object may be a vehicle, a human body, furniture, etc. The target detection image refers to an image containing the target object obtained by processing the image to be detected.

[0067] Specifically, the server can obtain an image to be detected by photographing the target scene through a photographing device (such as a camera), or use an existing image data set as the image to be detected. The server performs target detection processing on the image to be detected to obtain a target detection result of the image to be detected. If the target detection result indicates that there is a target object in the image to be detected, the image to be detected is subjected to image shearing processing according to the target detection result to obtain a target detection image; otherwise, the target detection processing is continued on the next image to be detected.

[0068] Step S202: outputting the integrity assessment result for the target object in the target detection image through the trained integrity assessment model.

[0069] Among them, the integrity assessment model is obtained by training the sample image obtained based on the shearing process and the integrity label corresponding to the sample image; the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

[0070] The integrity evaluation model is used to evaluate the integrity of the target object in the input image (such as the target detection image). The integrity evaluation result is used to characterize the integrity of the target object in the target detection image. The integrity label can be set to any value within the range of 1 to 100, for example, the integrity label can be set to 50, 90, 100, etc.

[0071] Specifically, the server may pre-cut the initial sample image according to different cutting ratios to obtain sample images of different completeness and completeness labels corresponding to the sample images. Then, the sample images and the completeness labels corresponding to the sample images are used to iteratively train the initial completeness assessment model to be trained, and finally a trained completeness assessment model is obtained. After obtaining the target detection image, the server may input the target detection image into the trained completeness assessment model to perform completeness assessment processing on the target object in the target detection image through the completeness assessment model, and finally output the completeness assessment result for the target object in the target detection image.

[0072] In the above image processing method, a target detection image of a target object in an image to be detected is obtained; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object; and the integrity evaluation result for the target object in the target detection image is output through a trained integrity evaluation model; wherein the integrity evaluation model is obtained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing process. By adopting this method, the integrity evaluation model obtained by training sample images of different integrity levels is used to realize the judgment of the integrity of the target object in the target detection image, and the accuracy of the evaluation of the integrity of the target object is effectively improved.

[0073] In one embodiment, in step S202, before outputting the integrity assessment result for the target object in the target detection image through the trained integrity assessment model, it also includes: performing image shearing processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image; inputting the sample image into the initial integrity assessment model to be trained to obtain the predicted integrity of the sample image; obtaining the loss value of the initial integrity assessment model based on the predicted integrity and the integrity label corresponding to the sample image; and iteratively updating the initial integrity assessment model based on the loss value to obtain the integrity assessment model.

[0074] Specifically, the server performs a rectangular cropping process on the initial sample image according to a randomly selected cropping ratio to obtain a cropped sample image, and sets the cropping ratio as the integrity label corresponding to the sample image. The server inputs the sample image into the initial integrity assessment model to be trained, and outputs the predicted integrity of the sample image; then, according to the integrity difference between the predicted integrity and the integrity label corresponding to the sample image, the loss value of the initial integrity assessment model is calculated; based on the loss value, the model parameters of the initial integrity assessment model are iteratively updated so that the predicted integrity output by the initial integrity assessment model is continuously close to the actual integrity label of the sample image, thereby improving the integrity prediction accuracy of the initial integrity assessment model, and finally obtaining a trained integrity assessment model.

[0075] In practical applications, the initial integrity assessment model can be a classification model, and the integrity label can be regarded as a category. Then, the initial integrity assessment model can be constructed through a classification network, such as using Resnet (deep residual network), MobileNet (a lightweight convolutional neural network) or CNN (Convolutional Neural Networks). The server can store sample images with the same integrity label in the same folder and set the integrity label to the name of the folder. The server then reads the sample images and folder names in each folder to train the initial integrity assessment model.

[0076] In this embodiment, the cut sample image is input into the initial integrity assessment model to be trained to obtain the predicted completeness of the sample image; the loss value of the initial integrity assessment model is obtained according to the completeness difference between the predicted completeness and the completeness label corresponding to the sample image; based on the loss value, the initial integrity assessment model is trained so that the predicted completeness output by the trained integrity assessment model continuously approaches the actual completeness label of the sample image, thereby improving the integrity assessment model's ability to assess the integrity of the target object in the image.

[0077] In one embodiment, Figure 3 As shown, the initial sample image is cut to obtain the sample image and the integrity label corresponding to the sample image, which specifically includes the following contents:

[0078] Step S301, determining the cropping ratio of the initial sample image.

[0079] The shearing ratio is used to represent the ratio of the number of pixels of the sample image after shearing to the number of pixels of the initial sample image.

[0080] It should be noted that the initial sample image is an image that contains the complete target object, and an image without the target object cannot be selected as the initial sample image. For example, the target object appears at the edge of the camera, resulting in only part of the target object being captured, or there is an occlusion that causes the target object to be partially or completely blocked.

[0081] Specifically, the server may randomly select a cropping range within a preset cropping range (eg, 1% to 100%) as the cropping ratio of the initial sample image, for example, 56% is selected as the cropping ratio of the initial sample image.

[0082] Step S302: obtaining the cropping size of the initial sample image according to the cropping ratio.

[0083] The shearing size is used to characterize the shearing start and end point positions and the shearing amplitude of the initial sample image.

[0084] Specifically, before the server performs a rectangular shearing process on the initial sample image, it needs to determine the shearing size first, determine the width and height of the initial sample image to be sheared according to the shearing ratio, and finally determine the specific shearing start and end points.

[0085] Step S303 , the width and height of the initial sample image are sheared according to the shearing size to obtain a sample image, and the shearing ratio is set as the integrity label corresponding to the sample image.

[0086] Specifically, the server performs cropping processing on the width and height of the initial sample image according to the cropping size, and finally obtains the cropped sample image. At the same time, the cropping ratio in the above step S301 can be set as the integrity label corresponding to the sample image.

[0087] In practical applications, the same initial sample image can be cut only once, and the cut sample image is placed in a folder whose folder name matches the completeness label; for example, a sample image with a completeness label of 56% is placed in a folder named "56", and a sample image with a completeness label of 83% is placed in a folder named "83". After placing the sample image in the folder, the server continues to process the next initial sample image. After all the initial sample images are cut, the server obtains the sample images required for training the initial completeness evaluation model.

[0088] It is understandable that the number of sample images in each folder can be different. The server can obtain a large number (for example, 1 million) of open-source initial sample images, or collect a large number of initial sample images in the target scene. The more initial sample images, the better. First, it helps to improve the training effect of the initial integrity assessment model. Second, when the number is large enough, even if the cropping ratio is randomly selected, the number of selected images for each cropping ratio will not differ much, which can ensure that the number of sample images corresponding to each integrity label is relatively balanced.

[0089] In this embodiment, the cropping size of the initial sample image is obtained according to the cropping ratio of the initial sample image, and then the initial sample image is cropped according to the cropping size to obtain the sample image, and the cropping ratio is set as the integrity label corresponding to the sample image, thereby effectively acquiring sample images of different completenesses without deliberately collecting images of different completenesses of the target object, and the integrity labels of the sample images obtained by the cropping process are also more accurate, thereby reducing the training cost of the model.

[0090] In one embodiment, the above step S302 obtains the cropping size corresponding to the initial sample image according to the cropping ratio, which specifically includes the following contents: according to the cropping ratio, obtain the target width information and the target height information of the initial sample image; according to the target width information, obtain the first cropping starting point and the first cropping end point of the initial sample image in width; according to the target height information, obtain the second cropping starting point and the second cropping end point of the initial sample image in height; according to the first cropping starting point, the first cropping end point, the second cropping starting point and the second cropping end point, obtain the cropping size.

[0091] The target width information indicates the width of the cropping range of the initial sample image, and also indicates the width of the sample image. The target height information indicates the height of the cropping range of the initial sample image, and also indicates the height of the sample image.

[0092] Specifically, the server randomly selects the target width information of the initial sample image according to the cropping ratio and the preset cropping range; wherein the target width information is greater than or equal to the cropping ratio, and the target width information is less than the upper limit of the preset cropping range. Then, according to the cropping ratio and the target width information, the target height information of the initial sample is obtained, which can be the result of dividing the cropping ratio by the target width information as the target height information. Then, the server selects the first cropping starting point on the width of the initial sample image, and then obtains the first cropping end point on the width according to the sum of the first cropping starting point and the target width information; similarly, the server randomly selects the second cropping starting point on the height of the initial sample image, and then obtains the second cropping end point on the height according to the sum of the second cropping starting point and the target height information. The server sets the first cropping starting point, the first cropping end point, the second cropping starting point, and the second cropping end point as the cropping size of the initial sample image; then, the subsequent server can crop an area with a size of the target width information * the target height information from the first cropping starting point, the first cropping end point, the second cropping starting point, and the second cropping end point of the initial sample image, and return the sample image of the cropping area.

[0093] For example:

[0094] 1) Randomly select a crop ratio between [1, 100), for example, set 56% as the crop ratio;

[0095] 2) The target width information is randomly selected from 0.56 to 1, that is, the target width information w = random[0.56, 1); the target height information h = 0.56 / w;

[0096] 3) Randomly select the cutting starting point of the x-axis from (0, 1-w), that is, the first cutting starting point of the initial sample image in width is x0 = random(0, 1-w), and the first cutting end point is x1=x0+w;

[0097] 4) Randomly select the cutting starting point of the y-axis from (0, 1-h), that is, the second cutting starting point y0 = random(0, 1-h) of the initial sample image at height, then the second cutting end point is y1=y0+h;

[0098] 5) x0, x1, y0, y1, w, h are the cropping percentages. They need to be multiplied by the width and height of the initial sample image to convert the percentages into the absolute values ​​of the initial sample image X0, X1, Y0, Y1, W, H. Then, the initial sample image is cropped according to X0, X1, Y0, Y1, W, H to obtain a sample image of the corresponding cropping size.

[0099] In this embodiment, the target width information and target height information of the initial sample image are determined according to the cropping ratio; then, the first cropping start point and the first cropping end point of the initial sample image in width are obtained according to the target width information, and the second cropping start point and the second cropping end point of the initial sample image in height are obtained according to the target height information, thereby determining the cropping size of the initial sample image, so that the subsequent steps can be based on the cropping size to perform cropping processing on the initial sample image, thereby obtaining sample images of different completeness, effectively reducing the training cost of the model.

[0100] In one embodiment, the loss value of the initial completeness assessment model is obtained based on the predicted completeness and the completeness label, which specifically includes the following contents: determining the probability distribution information and classification information corresponding to the predicted completeness; and obtaining the loss value of the initial completeness assessment model based on the probability distribution information and classification information.

[0101] Specifically, the server can determine the probability distribution information and classification information according to the predicted completeness, and then use the probability distribution information and classification information to calculate the loss value of the initial completeness evaluation model. In practical applications, the cross entropy loss function can be used to calculate the loss value of the initial completeness evaluation model. The formula of the cross entropy loss function is as follows:

[0102]

[0103] Where Loss represents the loss value; C represents the completeness label; p=[p0, p1, ..., p C-1 ] represents the probability distribution information, each element p i represents the probability that the sample image belongs to the i-th class integrity label; y=[y0, y1, ..., y C-1 ] is the classification information represented by the onehot encoding of the completeness label. When the sample image belongs to the completeness label i, y i =1, otherwise y i =0.

[0104] In this embodiment, the cross-entropy loss value of the initial completeness assessment model is calculated by predicting the probability distribution information and classification information corresponding to the completeness, so as to evaluate the difference between the predicted completeness output by the initial completeness assessment model and the actual completeness label through the cross-entropy loss value, thereby adjusting the performance of the initial completeness assessment model and improving the accuracy of the completeness assessment model in judging the completeness of the target object.

[0105] In one embodiment, the above step S201, obtaining a target detection image of the target object in the image to be detected, specifically includes the following contents: performing target detection processing on the image to be detected through a trained target detection model to obtain a target detection result for the target object; performing image shearing processing on the image to be detected according to the target detection result to obtain a target detection image.

[0106] The target detection model is used to identify specific objects (such as target objects) in the image. The target detection result is used to characterize the location of the target object in the image to be detected.

[0107] Specifically, the server can pre-train a target detection model. After acquiring the image to be detected, the image to be detected is input into the target detection model to detect whether there is a target object in the image to be detected through the target detection model, and obtain a target detection result for the target object; if the target detection result indicates that there is a target object in the image to be detected, then according to the position information of the target object in the image to be detected, the area containing the target object in the image to be detected is extracted, and the remaining area in the image to be detected is filtered out as background information, and finally the target detection image is output.

[0108] In this embodiment, target detection processing is performed on the image to be detected through the target detection model to obtain the target detection result, and then the target detection image is extracted from the image to be detected based on the target detection result. Then, the subsequent steps can directly perform integrity assessment on the target detection image, which effectively reduces the amount of data calculation and improves the efficiency of integrity assessment of the target object; background information in the image to be detected is removed, and the interference of background information on the integrity assessment can be avoided, thereby improving the accuracy of the integrity assessment of the target object in the target detection image.

[0109] In one embodiment, after obtaining the integrity evaluation result of the target object in step S202, the method further includes: determining a target processing image that meets a preset integrity condition from a plurality of target detection images to be detected based on the integrity evaluation result; and performing recognition processing on the target processing image.

[0110] The preset integrity condition refers to a pre-set judgment condition for the integrity evaluation result. For example, the preset integrity condition can be set to exceed the integrity threshold.

[0111] Specifically, the server determines a target processing image that meets a preset completeness condition from the target detection images of multiple images to be detected based on the completeness evaluation results of each target detection image. The server may screen out target detection images whose completeness evaluation results reach and / or exceed the completeness threshold and set them as the target processing images, or screen out target detection images with the highest completeness evaluation results and set them as the target processing images. The server then uses the target processing image for subsequent recognition processing.

[0112] In this embodiment, based on the completeness evaluation result, a target processing image that meets the preset completeness condition can be determined from the target detection images of multiple images to be detected to perform recognition processing, which effectively improves the data quality of the image used for recognition processing and thus improves the accuracy of the recognition processing.

[0113] In one embodiment, Figure 4 As shown, another image processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0114] Step S401 : performing image shearing processing on an initial sample image to obtain a sample image and an integrity label corresponding to the sample image.

[0115] Step S402: input the sample image into the initial completeness evaluation model to be trained to obtain the predicted completeness of the sample image.

[0116] Step S403, obtaining the loss value of the initial integrity assessment model according to the predicted integrity and the integrity label corresponding to the sample image.

[0117] Step S404: iteratively update the initial integrity assessment model based on the loss value to obtain the integrity assessment model.

[0118] Step S405 , obtaining a target detection image of the target object in the image to be detected; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object.

[0119] Step S406: outputting the integrity evaluation result for the target object in the target detection image through the trained integrity evaluation model.

[0120] Step S407, based on the integrity evaluation result, determine a target processing image that meets a preset integrity condition from the target detection images of the multiple images to be detected; and perform recognition processing on the target processing image.

[0121] The above-mentioned image processing method can achieve the following beneficial effects: by using the integrity assessment model trained by sample images of different completeness, the integrity of the target object in the target detection image can be judged, and the accuracy of the assessment of the integrity of the target object can be effectively improved.

[0122] In order to more clearly illustrate the image processing method provided by the embodiment of the present disclosure, the above-mentioned image processing method is specifically described below with a specific embodiment. Another image processing method is provided, which can be applied to Figure 1 The server in the system includes the following:

[0123] Figure 5 A schematic diagram of the process of obtaining the cut vehicle image is shown in Figure 1. Figure 5 As shown, the server can collect a large number (for example, 1 million) of vehicle images, in each of which the vehicle is completely captured. Then, each vehicle image is subjected to a rectangular shearing process with a random shearing ratio from 1% to 100% to obtain a sheared vehicle image, and the shearing ratio is set as the integrity label of the sheared vehicle image. A folder with the same name is created for each integrity label from 1 to 100, and the sheared vehicle image is placed in a folder with a corresponding name according to the integrity label of the sheared vehicle image, for example, a sheared vehicle image with a integrity label of 1 is placed in a folder with a name of 1, and a sheared vehicle image with a integrity label of 52 is placed in a folder with a name of 52. Finally, the server obtains 1 million sheared vehicle images. Figure 6 The figure is a flowchart of the training process of the completeness evaluation model. The server can select the ResNet network to build the initial completeness evaluation model, and use the cross entropy loss function as the loss function of the initial completeness evaluation model. The cropped vehicle image is input into the initial completeness evaluation model, and the predicted completeness logits_label of the cropped vehicle image is output. The initial completeness evaluation model is continuously trained according to the difference between the predicted completeness and the completeness label, so that the predicted completeness logits_label output by the model is continuously close to the actual completeness label gt_label. After multiple rounds of training, the completeness evaluation model is obtained.

[0124] In traffic scenes, when a vehicle passes by, multiple snapshots (i.e., images to be detected) may be captured. If vehicle recognition processing is performed on each snapshot, the data processing volume is large and the recognition efficiency is low. Second, some snapshots are of low quality and the vehicles in the images are incomplete, so it is not meaningful to perform recognition processing on them. In this case, the background information in the snapshot can be removed first, and the vehicle area image (i.e., target detection image) can be extracted. Figure 7The diagram is a schematic diagram of the image to be detected and the target detection image. Then, the integrity of the vehicles in the vehicle area image is evaluated respectively by the integrity evaluation model to obtain the integrity evaluation result of the vehicle area image, and then, according to the integrity evaluation result, a vehicle area image with the highest integrity can be selected for subsequent vehicle recognition processing.

[0125] In this embodiment, there is no need to collect additional data sets to train the initial integrity assessment model. The data sets can be generated by using the captured images in the actual target scene, and the model training cost is low. Then, the integrity assessment model trained by the cropped vehicle images with different integrity can be used to accurately judge the integrity of the vehicle in the vehicle area image.

[0126] In one embodiment, Figure 1 As shown, an image processing system is provided, which includes a shooting device and a server; the shooting device is used to collect an image to be detected of a target scene and send it to the server; the server is used to obtain a target detection image of the target object in the received image to be detected; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object; through a trained integrity assessment model, an integrity assessment result for the target object in the target detection image is output; wherein the integrity assessment model is obtained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

[0127] The target scene refers to the scene that needs to be detected and identified. The target scene includes but is not limited to smart transportation scenes, smart home scenes, etc.

[0128] The shooting device is used to collect the image to be detected of the target scene. The shooting device can be a monitoring device, camera, smart door lock or other device with shooting function installed in the target scene.

[0129] Specifically, after the shooting device captures the image to be detected of the target scene, the image to be detected is sent to the server, and the server performs subsequent processing on the image to be detected. The specific processing process is shown in the above embodiment and will not be repeated here.

[0130] In this embodiment, the image to be detected of the target scene is collected by a shooting device and sent to a server, and then the server extracts a target detection image containing the target object from the received image to be detected, and then uses the trained integrity assessment model to output the integrity assessment result of the target object in the target detection image, thereby achieving effective judgment on the integrity of the target object in the image and improving the accuracy of judgment on the target object.

[0131] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0132] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0133] In one embodiment, Figure 8 As shown, an image processing device 800 is provided, comprising: an image detection module 801 and an image evaluation module 802, wherein:

[0134] The image detection module 801 is used to obtain a target detection image of a target object in the image to be detected; the target detection image is obtained by processing the image to be detected according to the target detection result of the target object.

[0135] The image evaluation module 802 is used to output the integrity evaluation result of the target object in the target detection image through the trained integrity evaluation model; wherein the integrity evaluation model is obtained by training the sample image obtained based on the shearing process and the integrity label corresponding to the sample image; the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

[0136] In one embodiment, the image processing device 800 also includes a model training module, which is used to perform image cropping processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image; input the sample image into the initial integrity assessment model to be trained to obtain the predicted integrity of the sample image; obtain the loss value of the initial integrity assessment model based on the predicted integrity and the integrity label corresponding to the sample image; based on the loss value, iteratively update the initial integrity assessment model to obtain the integrity assessment model.

[0137] In one embodiment, the image processing device 800 also includes an image cropping module, which is used to determine the cropping ratio of the initial sample image; obtain the cropping size of the initial sample image according to the cropping ratio; crop the width and height of the initial sample image according to the cropping size to obtain the sample image, and set the cropping ratio as the integrity label corresponding to the sample image.

[0138] In one embodiment, the image processing device 800 also includes a size determination module, which is used to obtain target width information and target height information of the initial sample image according to the cropping ratio; obtain a first cropping starting point and a first cropping end point in width of the initial sample image according to the target width information; obtain a second cropping starting point and a second cropping end point in height of the initial sample image according to the target height information; and obtain a cropping size according to the first cropping starting point, the first cropping end point, the second cropping starting point, and the second cropping end point.

[0139] In one embodiment, the image processing device 800 further includes a loss value obtaining module for determining probability distribution information and classification information corresponding to the predicted completeness; and obtaining the loss value of the initial completeness assessment model based on the probability distribution information and classification information.

[0140] In one embodiment, the image detection module 801 is also used to perform target detection processing on the image to be detected through a trained target detection model to obtain a target detection result for the target object; based on the target detection result, perform image shearing processing on the image to be detected to obtain a target detection image.

[0141] In one embodiment, the image processing device 800 further includes an image recognition module for determining a target processing image that meets a preset integrity condition from target detection images of multiple images to be detected based on the integrity evaluation result; and performing recognition processing on the target processing image.

[0142] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0143] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as images to be detected and target detection images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0144] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0147] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object; Outputting, by means of a trained integrity assessment model, an integrity assessment result for the target object in the target detection image; The integrity assessment model is obtained by training a sample image obtained through shearing processing and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing processing.

2. The method according to claim 1, characterized in that Before outputting the integrity evaluation result for the target object in the target detection image through the trained integrity evaluation model, the method further includes: Performing image shearing processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image; Inputting the sample image into an initial completeness assessment model to be trained to obtain a predicted completeness of the sample image; Obtaining a loss value of the initial integrity assessment model according to the predicted integrity and the integrity label corresponding to the sample image; Based on the loss value, the initial integrity assessment model is iteratively updated to obtain the integrity assessment model.

3. The method according to claim 2, characterized in that The performing image shearing processing on the initial sample image to obtain the sample image and the integrity label corresponding to the sample image includes: Determining the shearing ratio of the initial sample image; According to the shearing ratio, obtaining a shearing size of the initial sample image; The width and height of the initial sample image are sheared according to the shearing size to obtain the sample image, and the shearing ratio is set as the integrity label corresponding to the sample image.

4. The method according to claim 3, characterized in that The step of obtaining a cropping size corresponding to the initial sample image according to the cropping ratio includes: According to the shearing ratio, obtaining target width information and target height information of the initial sample image; According to the target width information, obtaining a first cutting start point and a first cutting end point of the initial sample image at the width; According to the target height information, obtain a second shearing starting point and a second shearing end point of the initial sample image at the height; The shearing size is obtained according to the first shearing starting point, the first shearing end point, the second shearing starting point and the second shearing end point.

5. The method according to claim 2, characterized in that: The obtaining, according to the predicted completeness and the completeness label, a loss value of the initial completeness assessment model comprises: Determining probability distribution information and classification information corresponding to the prediction completeness; A loss value of the initial integrity assessment model is obtained according to the probability distribution information and the classification information.

6. The method according to claim 1, characterized in that The step of obtaining a target detection image of a target object in an image to be detected includes: Performing target detection processing on the image to be detected by using the trained target detection model to obtain a target detection result for the target object; According to the target detection result, image shearing processing is performed on the image to be detected to obtain the target detection image.

7. The method according to any one of claims 1 to 6, characterized in that: After obtaining the integrity evaluation result of the target detection image, the method further includes: Based on the completeness evaluation result, determining a target processing image that meets a preset completeness condition from the target detection images of the plurality of images to be detected; Perform recognition processing on the target processing image.

8. An image processing device, characterized in that: The device comprises: An image detection module is used to obtain a target detection image of a target object in an image to be detected; the target detection image is obtained by processing the image to be detected according to a target detection result of the target object; An image evaluation module is used to output an integrity evaluation result for the target object in the target detection image through a trained integrity evaluation model; wherein the integrity evaluation model is trained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; and the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

9. An image processing system, characterized in that: The system includes a shooting device and a server; The shooting device is used to collect the image to be detected of the target scene and send it to the server; The server is used to obtain a target detection image of the target object in the received image to be detected; The target detection image is obtained by processing the image to be detected according to the target detection result of the target object; the integrity assessment result of the target object in the target detection image is output through a trained integrity assessment model; wherein the integrity assessment model is obtained by training a sample image obtained based on a shearing process and an integrity label corresponding to the sample image; the integrity label is obtained based on the shearing ratio of the sample image during the shearing process.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images