Image processing method and device and refrigeration equipment

Abstract

The invention discloses an image processing method and device and refrigeration equipment, and belongs to the technical field of image processing. The image processing method comprises the following steps: acquiring a to-be-corrected image corresponding to a to-be-detected object; based on at least one piece of angular point information in the multiple pieces of angular point information corresponding to each chamber, performing segmentation processing on the to-be-corrected image to obtain at least one first image corresponding to the to-be-corrected image; and performing correction processing on each first image to obtain a target detection image corresponding to the to-be-detected object. According to the image processing method, errors and information loss possibly generated by one-time correction are effectively reduced, a distortionless complete image can also be obtained when the shooting angle is large or distortion is serious, and then the precision and accuracy of object detection and recognition based on the target detection image are improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to an image processing method, device and refrigeration equipment. Background Art

[0002] When photographing the merchandise displayed in the freezer, the captured images often have significant angle deviation and distortion, which causes image distortion and deformation, thus affecting the accuracy of object detection and recognition. In the related art, there are methods to improve image quality through geometric correction, but this method cannot effectively correct images with large shooting angles or severe distortion, resulting in low accuracy of object detection and recognition. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an image processing method, device and refrigeration equipment, which effectively reduces the error and information loss that may be caused by a correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0004] In a first aspect, the present application provides an image processing method, the method comprising:

[0005] Acquire an image to be corrected corresponding to the object to be measured, wherein the image to be corrected includes information of multiple corner points corresponding to multiple compartments in the refrigeration equipment, and the compartments are used to store the object to be measured;

[0006] Based on at least one corner point information among the multiple corner point information corresponding to each of the compartments, segment the image to be corrected to obtain at least one first image corresponding to the image to be corrected;

[0007] Correction processing is performed on each of the first images to obtain a target detection image corresponding to the object to be detected.

[0008] According to the image processing method provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle room of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0009] In an image processing method according to an embodiment of the present application, the correction processing is performed on each of the first images to obtain a target detection image corresponding to the object to be detected, including:

[0010] Based on at least one corner point information in each of the first images, respectively perform correction processing on each of the first images to obtain a second image corresponding to each of the first images;

[0011] Perform stitching processing on at least one of the second images to obtain the target detection image.

[0012] An image processing method according to an embodiment of the present application, wherein the correction processing is performed on each of the first images based on at least one corner point information in each of the first images to obtain a second image corresponding to each of the first images, comprises:

[0013] Based on at least one corner point information in each of the first images and the to-be-aligned corner point information corresponding to each of the corner point information, acquiring a perspective transformation matrix between the first image and a second image corresponding to the first image;

[0014] Based on the perspective transformation matrix, perspective transformation is performed on the first image to obtain a second image corresponding to the first image.

[0015] In an image processing method according to an embodiment of the present application, performing perspective transformation on the first image based on the perspective transformation matrix to obtain a second image corresponding to the first image includes:

[0016] Based on the perspective transformation matrix, the first image at the first viewing angle is perspective transformed to a second viewing angle to obtain a second image corresponding to the first image.

[0017] In an image processing method according to an embodiment of the present application, the stitching process of at least one of the second images to obtain the target detection image includes:

[0018] Each of the second images is converted into an image of the same size, and at least one of the converted second images is spliced ​​to obtain the target detection image.

[0019] In an image processing method according to an embodiment of the present application, the stitching process of at least one of the second images to obtain the target detection image includes:

[0020] Acquire common corner point information between a target second image in at least one of the second images and other second images in at least one of the second images except the target second image;

[0021] Based on the same corner point information, a first edge to be stitched in the target second image and a second edge to be stitched in the other second image are acquired; all corner point information corresponding to the first edge to be stitched corresponds to all corner point information corresponding to the second edge to be stitched in one-to-one correspondence;

[0022] The first edge to be spliced ​​and the second edge to be spliced ​​are spliced ​​to obtain the target detection image.

[0023] In an image processing method according to an embodiment of the present application, obtaining an image to be corrected corresponding to the object to be measured comprises:

[0024] Acquire at least one of an internal parameter and a distortion parameter corresponding to an image acquisition device used to acquire the image to be corrected;

[0025] Based on at least one of the internal parameter and the distortion parameter, the initial image corresponding to the object to be measured acquired by the image acquisition device is corrected to obtain the image to be corrected.

[0026] In a second aspect, the present application provides an image processing device, comprising:

[0027] A first processing module is used to obtain an image to be corrected corresponding to the object to be measured, wherein the image to be corrected includes information of multiple corner points corresponding to multiple compartments in the refrigeration equipment, and the compartments are used to store the object to be measured;

[0028] A second processing module, configured to segment the image to be corrected based on at least one corner point information among the plurality of corner point information corresponding to each of the compartments, to obtain at least one first image corresponding to the image to be corrected;

[0029] The third processing module is used to perform correction processing on each of the first images respectively to obtain a target detection image corresponding to the object to be detected.

[0030] According to the image processing device provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle room of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0031] In a third aspect, the present application provides a refrigeration device, comprising:

[0032] Box;

[0033] A plurality of compartments, each of which is installed in the box and is used to store objects to be tested;

[0034] As described in the second aspect, the image processing device is used to process the image to be corrected corresponding to the object to be measured.

[0035] According to the refrigeration equipment provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle chamber of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0036] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method as described in the first aspect above when executing the computer program.

[0037] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image processing method as described in the first aspect above is implemented.

[0038] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method as described in the first aspect above.

[0039] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0040] The image to be corrected is segmented using the corner point information in the middle room of the image to be corrected, and then the first image obtained by segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0041] Furthermore, by transforming each first image individually and then stitching the second images obtained by the independent transformation, a more accurate target detection image under a bird's-eye view can be obtained, thereby improving the precision and accuracy of image correction.

[0042] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 This is one of the flowcharts of the image processing method provided in the embodiment of the present application;

[0045] Figure 2 This is the second flowchart of the image processing method provided in the embodiment of the present application;

[0046] Figure 3 This is one of the principle schematic diagrams of the image processing method provided in the embodiment of the present application;

[0047] Figure 4 This is the second principle schematic diagram of the image processing method provided in the embodiment of the present application;

[0048] Figure 5 This is the third principle schematic diagram of the image processing method provided in the embodiment of the present application;

[0049] Figure 6 This is the fourth principle schematic diagram of the image processing method provided in the embodiment of the present application;

[0050] Figure 7 This is the fifth principle schematic diagram of the image processing method provided in the embodiment of the present application;

[0051] Figure 8 This is the sixth principle schematic diagram of the image processing method provided in the embodiment of the present application;

[0052] Fig. 9 This is the third flowchart of the image processing method provided in the embodiment of the present application;

[0053] Fig.10 This is one of the result schematic diagrams of the image processing method provided in the embodiment of the present application;

[0054] Fig.11 This is the second result schematic diagram of the image processing method provided in the embodiment of the present application;

[0055] Fig.12 This is the third result schematic diagram of the image processing method provided in the embodiment of the present application;

[0056] Fig.13 This is the fourth result schematic diagram of the image processing method provided in the embodiment of the present application;

[0057] Fig.14 This is the fifth result schematic diagram of the image processing method provided in the embodiment of the present application;

[0058] Fig.15 This is the sixth result schematic diagram of the image processing method provided in the embodiment of the present application;

[0059] Fig.16 This is the seventh result schematic diagram of the image processing method provided in the embodiment of the present application;

[0060] Fig.17 This is the eighth result schematic diagram of the image processing method provided in the embodiment of the present application;

[0061] Fig.18 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0062] Fig.19 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0064] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0065] The image processing method, image processing device, electronic device and readable storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0066] The image processing method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0067] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0068] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0069] The image processing method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the image processing method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The image processing method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.

[0070] like Figure 1 As shown, the image processing method includes: step 110, step 120 and step 130.

[0071] The image processing method can be applied to refrigeration equipment, wherein refrigeration equipment can be understood as refrigeration storage equipment in a broad sense, including but not limited to refrigerators, freezers, display cabinets, beverage cabinets, wine cabinets, fresh-keeping cabinets, refrigerated vending machines and other refrigeration storage equipment. Refrigeration equipment has various structural forms and a wide range of applications.

[0072] Step 110: Obtain the image to be corrected corresponding to the object to be measured.

[0073] In this step, an image of the interior of the refrigeration equipment may be acquired by an image acquisition device (such as a camera).

[0074] For example, in a commercial refrigerator monitoring scenario, the image acquisition device may be a wide-angle camera, which may be installed on the side of the refrigerator to capture information on the product display surface.

[0075] When the image inside the refrigeration equipment is collected using a wide-angle camera, due to the lateral installation position of the camera and the characteristics of the wide-angle lens, the collected image may have obvious angle deviation and lens distortion.

[0076] The image to be corrected is an image that needs to be corrected to remove the distortion of the image to be corrected.

[0077] The image to be corrected includes information of a plurality of corner points corresponding to a plurality of compartments in the refrigeration equipment.

[0078] The refrigeration device includes a plurality of compartments, wherein the compartments are used to store the objects to be tested, and the compartments may include cargo frames.

[0079] The multiple compartments may be arranged in a straight line, and the multiple compartments may form multiple horizontal or vertical straight lines.

[0080] Each compartment may correspond to a plurality of corner point information, for example, the plurality of corner point information may include the upper left corner, the upper right corner, the lower left corner and the lower right corner of the compartment.

[0081] Corner points are points in the image where the edge changes dramatically, such as the edge intersection of the compartment.

[0082] The object to be tested is an object or article stored in the refrigeration equipment.

[0083] The object to be detected is an object that needs to be detected, identified or analyzed.

[0084] In some embodiments, after obtaining the image to be corrected corresponding to the object to be measured, the method may include:

[0085] A key point detection algorithm is used to obtain multiple corner point information corresponding to multiple compartments in the image to be corrected.

[0086] In this embodiment, a key point detection algorithm can locate specific points of an object in the image to be rectified.

[0087] Key point detection algorithms may include SIFT (Scale-Invariant Feature Transform) algorithm and YOLO algorithm (You Only Look Once, target detection algorithm), etc.

[0088] For example, the YOLOv8 key point detection algorithm can be used to locate multiple corner point information of each compartment (such as a refrigerator cargo box).

[0089] like Fig.10 As shown, each compartment can be detected and marked separately, for example, a rectangular frame, a circular frame or other shapes can be used to mark the boundaries of the compartment, and a point or a cross mark can be used to mark the corner point position.

[0090] In the present application, the corner positions of the compartments are accurately located through a key point detection algorithm, so that the key point detection accuracy can be guaranteed even when the image to be corrected is distorted or the image is large.

[0091] Step 120: Based on at least one corner point information among the multiple corner point information corresponding to each compartment, the image to be corrected is segmented to obtain at least one first image corresponding to the image to be corrected.

[0092] In this step, if Fig.11 As shown, at least one corner point information among a plurality of corner point information can be used to determine the position and range of each compartment, and then the image to be corrected can be segmented into at least one first image.

[0093] Each first image may correspond to one or more compartments.

[0094] For example, when the distortion degrees of two adjacent compartments in the image to be corrected are the same, the two adjacent compartments may be segmented into the same first image when segmenting the image to be corrected.

[0095] The sizes of the first images may be different, and the distortion degrees of the first images may also be different.

[0096] The number of the at least one first image may be one or more.

[0097] Step 130: perform correction processing on each first image to obtain a target detection image corresponding to the object to be detected.

[0098] In this step, each first image may be corrected, and then a target detection image may be acquired based on the multiple corrected images.

[0099] The target detection image includes characteristic information of the object to be detected. Based on the target detection image, the characteristic information of the object to be detected can be identified and extracted so that the user can obtain the storage status in the refrigeration equipment.

[0100] For example, in a commercial refrigerator monitoring scenario, after obtaining the target detection image, the display situation of the goods in the refrigeration equipment can be obtained based on the target detection image. When there are fewer goods in a certain category, the user can be reminded to replenish the goods in that category in a timely manner.

[0101] like Figure 2 As shown, in some embodiments, after acquiring the target detection image, the target detection image can be saved to a database or OSS (Object Storage Service).

[0102] For example, when it is necessary to obtain the storage status of items in a refrigeration device, a target detection image can be obtained from a database or OSS, and image feature extraction and other processing can be performed on the target detection image.

[0103] According to the image processing method provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle room of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0104] like Figure 2 As shown, in some embodiments, step 130 may include:

[0105] Based on at least one corner point information in each first image, correct each first image to obtain a second image corresponding to each first image;

[0106] The at least one second image is stitched to obtain a target detection image.

[0107] In this embodiment, the second image is an image obtained after correction processing is performed on the first image.

[0108] Correction processing may include rotation, scaling, and translation processing, etc.

[0109] For example, a perspective transformation may be performed on the first image to obtain a second image corresponding to the first image.

[0110] After the correction process is performed on the at least one first image, the at least one second image obtained by the correction can be spliced.

[0111] At least one second image may be spliced ​​based on an arrangement order of at least one first image.

[0112] Feature points of each second image may be extracted and matched with feature points of adjacent images, and then at least one second image may be spliced ​​based on the matched feature points.

[0113] When stitching the second image, the stitched images may be fused to eliminate problems such as seams and brightness inconsistency at the stitching locations. For example, a weighted average or Laplace pyramid fusion algorithm may be used to perform image fusion.

[0114] In some embodiments, based on at least one corner point information in each first image, performing correction processing on each first image to obtain a second image corresponding to each first image may include:

[0115] Based on at least one corner point information in each first image and the to-be-aligned corner point information corresponding to each corner point information, obtaining a perspective transformation matrix between the first image and a second image corresponding to the first image;

[0116] Based on the perspective transformation matrix, a perspective transformation is performed on the first image to obtain a second image corresponding to the first image.

[0117] In this embodiment, the corner points to be aligned are the corner points that need to be aligned among the extracted corner points.

[0118] For example, based on the user-defined corner point information of the position that the compartment should occupy in the second image, multiple corner point information to be aligned corresponding to each compartment can form a rectangle, which can be used to represent the shape of the compartment in the second image.

[0119] The perspective transformation matrix is ​​used to represent the perspective transformation relationship from the first image to the second image.

[0120] Based on at least one corner point information and the corner point information to be aligned corresponding to each corner point information, a linear equation group may be constructed and solved to calculate a perspective transformation matrix.

[0121] For example, the getPerspectiveTransform function of OpenCV can be used to calculate the perspective transformation matrix based on at least one corner point information and the corner point information to be aligned corresponding to each corner point information; the getPerspectiveTransform function can return a 3x3 transformation matrix, that is, the perspective transformation matrix.

[0122] The perspective transformation matrix may be applied to the first image to change the shape and size of the first image so as to make it consistent with the spatial position represented by the corner point information to be aligned.

[0123] When the perspective transformation is performed on the first image, an interpolation process may be performed to obtain the color value of each pixel position of the transformed image.

[0124] In the actual execution process, the warpPerspective function can be used to apply the calculated perspective transformation matrix, or other functions can be used. The selection can be based on user needs and is not limited in this application.

[0125] In some embodiments, performing perspective transformation on the first image based on the perspective transformation matrix to obtain a second image corresponding to the first image may include:

[0126] Based on the perspective transformation matrix, the first image at the first viewing angle is perspective transformed to the second viewing angle to obtain a second image corresponding to the first image.

[0127] In this embodiment, the first viewing angle may be a side viewing angle, and the second viewing angle may be a top viewing angle.

[0128] like Figure 7 As described above, for each first image, the perspective transformation matrix can be used to transform the first image at a side view into a second image at a top view.

[0129] The shape of the first image may be a vertical trapezoid or a horizontal trapezoid, or may be other shapes, which is not limited in the present application.

[0130] like Figure 8 As shown, a first image in a vertical trapezoidal shape or a horizontal trapezoidal shape can be transformed into a second image in a rectangular shape.

[0131] In some embodiments, performing stitching processing on at least one second image to obtain a target detection image may include:

[0132] The second images are converted into images of the same size, and at least one of the converted second images is spliced ​​to obtain a target detection image.

[0133] In this embodiment, the sizes of the second images may be different, and at least one second image may be equidistantly transformed to convert the at least one second image into an image of the same size.

[0134] In some embodiments, performing stitching processing on at least one second image to obtain a target detection image may include:

[0135] Acquire common corner point information between a target second image in at least one second image and other second images in at least one second image except the target second image;

[0136] Based on the same corner point information, a first edge to be spliced ​​in the target second image and a second edge to be spliced ​​in the other second image are obtained;

[0137] The first edge to be spliced ​​and the second edge to be spliced ​​are spliced ​​to obtain a target detection image.

[0138] In this embodiment, the target second image is any image of the at least one second image.

[0139] The same corner point information between the target second image and other second images can be identified and obtained, and the same corner point information can be used for alignment processing between the images.

[0140] The edge to be stitched is the area in the image that needs to be stitched, for example, it may be the boundary of the image.

[0141] All corner point information corresponding to the first edge to be spliced ​​corresponds to all corner point information corresponding to the second edge to be spliced ​​in one-to-one correspondence.

[0142] like Fig. 9 As shown, when the target second image is image 2, and the other second images are image 1 and image 3, the same corner point information between image 1 and image 2: corner point B and corner point G, and the same corner point information between image 2 and image 3: corner point C and corner point F can be obtained;

[0143] Based on corner point B and corner point G, the first edge L11 to be stitched of image 2 and the second edge L12 to be stitched of image 1 can be obtained; based on corner point C and corner point F, the first edge L21 to be stitched of image 2 and the second edge L22 to be stitched of image 3 can be obtained;

[0144] Then, the edges to be spliced ​​corresponding to the corresponding corner point information can be spliced, that is, L11 and L12 can be spliced, and L21 and L22 can be spliced, and then the target detection image can be obtained.

[0145] According to the image processing method provided in the embodiment of the present application, by transforming each first image individually and then stitching the second images obtained by the independent transformation, a more accurate target detection image from a bird's-eye view can be obtained, thereby improving the precision and accuracy of image correction.

[0146] like Figure 2 As shown, in some embodiments, performing stitching processing on at least one second image to obtain a target detection image may include:

[0147] Performing splicing processing on at least one second image to obtain an initial detection image;

[0148] Perform a rigid transformation on the initial detection image to obtain the target detection image.

[0149] In this embodiment, a rigid transformation is performed on the image, that is, the image can be translated and rotated, and the shape and size of the object in the image can be kept unchanged.

[0150] In a rigid transformation, every point in the initial detection image is transformed by the same rotation angle and translation vector.

[0151] like Figure 2 As shown, in some embodiments, step 110 may include:

[0152] Acquiring at least one of an internal parameter and a distortion parameter corresponding to an image acquisition device for acquiring an image to be corrected;

[0153] Based on at least one of the internal parameters and the distortion parameters, the initial image corresponding to the object to be measured acquired by the image acquisition device is corrected to obtain an image to be corrected.

[0154] In this embodiment, the internal parameters are camera internal parameters, which are used to characterize the internal optical characteristics of the image acquisition device.

[0155] Intrinsic parameters may include focal length and principal point, etc.

[0156] Distortion parameters may include radial distortion (barrel distortion and pincushion distortion) and tangential distortion, etc.

[0157] The intrinsic parameters and distortion parameters can be obtained through camera calibration. For example, a calibration plate of known size can be used to take multiple images, and then the intrinsic parameters and distortion parameters can be calculated.

[0158] Based on the acquired internal parameters and distortion parameters, a correction algorithm may be selected to perform a geometric transformation on the initial image to eliminate the distortion caused by the distortion parameters.

[0159] The intrinsic parameters can be used to transform the image from the camera coordinate system to the image coordinate system and ensure that the image has the correct scale and position.

[0160] like Figure 2 As shown, a camera calibration tool (such as MATLAB's Camera Calibration Toolbox or OpenCV's calibrateCamera function) is used to capture multiple images containing a known checkerboard pattern, and based on these images, the internal parameters and distortion parameters of the image acquisition device are calculated.

[0161] For example, a checkerboard pattern of known dimensions may be created (eg, an 8x8 grid of squares, each square being 20 mm in size).

[0162] like Figures 3 to 6 As shown, a camera can be used to capture images containing checkerboard patterns from multiple angles to ensure that the checkerboard patterns occupy different positions and angles in the image. A camera calibration tool is then used to calibrate the camera, and the intrinsic parameters and distortion parameters of the camera are extracted from the calibration results.

[0163] After multiple tests by the inventors, the image processing method provided in the embodiment of the present application can correct the distorted image and can collect Fig.12 The image to be corrected as shown in FIG. 1 can be corrected by the image processing method described in any of the above embodiments to obtain the following image: Fig.13 The target detection image shown in Fig.14 Corrected to Fig.15 The image shown can be Fig.16 Corrected to Fig.17 Image shown.

[0164] The image processing device provided by the present application is described below. The image processing device described below and the image processing method described above can be referenced to each other.

[0165] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0166] The embodiment of the present application also provides an image processing device.

[0167] like Fig.18 As shown, the image processing device includes: a first processing module 1810, a second processing module 1820 and a third processing module 1830.

[0168] The first processing module 1810 is used to obtain an image to be corrected corresponding to the object to be measured, where the image to be corrected includes information of multiple corner points corresponding to multiple compartments in the refrigeration equipment, where the compartments are used to store the object to be measured;

[0169] The second processing module 1820 is used to segment the image to be corrected based on at least one corner point information among the multiple corner point information corresponding to each compartment, so as to obtain at least one first image corresponding to the image to be corrected;

[0170] The third processing module 1830 is used to perform correction processing on each first image respectively to obtain a target detection image corresponding to the object to be detected.

[0171] According to the image processing device provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle room of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0172] In some embodiments, the third processing module 1830 may also be used to:

[0173] Based on at least one corner point information in each first image, correct each first image to obtain a second image corresponding to each first image;

[0174] The at least one second image is stitched to obtain a target detection image.

[0175] In some embodiments, the third processing module 1830 may also be used to:

[0176] Based on at least one corner point information in each first image and the to-be-aligned corner point information corresponding to each corner point information, obtaining a perspective transformation matrix between the first image and a second image corresponding to the first image;

[0177] Based on the perspective transformation matrix, a perspective transformation is performed on the first image to obtain a second image corresponding to the first image.

[0178] In some embodiments, the third processing module 1830 may also be used to:

[0179] Based on the perspective transformation matrix, the first image at the first viewing angle is perspective transformed to the second viewing angle to obtain a second image corresponding to the first image.

[0180] In some embodiments, the third processing module 1830 may also be used to:

[0181] The second images are converted into images of the same size, and at least one of the converted second images is spliced ​​to obtain a target detection image.

[0182] In some embodiments, the third processing module 1830 may also be used to:

[0183] Acquire common corner point information between a target second image in at least one second image and other second images in at least one second image except the target second image;

[0184] Based on the same corner point information, a first edge to be stitched in the target second image and a second edge to be stitched in the other second image are obtained; all corner point information corresponding to the first edge to be stitched corresponds to all corner point information corresponding to the second edge to be stitched in one-to-one correspondence;

[0185] The first edge to be spliced ​​and the second edge to be spliced ​​are spliced ​​to obtain a target detection image.

[0186] In some embodiments, the first processing module 1810 may also be used to:

[0187] Acquiring at least one of an internal parameter and a distortion parameter corresponding to an image acquisition device for acquiring an image to be corrected;

[0188] Based on at least one of the internal parameters and the distortion parameters, the initial image corresponding to the object to be measured acquired by the image acquisition device is corrected to obtain an image to be corrected.

[0189] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0190] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0191] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 17 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0192] In some embodiments, Fig.19 As shown, an embodiment of the present application also provides an electronic device 1900, including a processor 1901, a memory 1902, and a computer program stored in the memory 1902 and executable on the processor 1901. When the program is executed by the processor 1901, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0193] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0194] An embodiment of the present application also provides a refrigeration device, including: a box body, a plurality of compartments and an image processing device.

[0195] In this embodiment, the compartment may be installed in the box, and the compartment is used to store the object to be tested.

[0196] The refrigeration device may further include an image acquisition device.

[0197] The image acquisition device may be installed above or obliquely above the multiple compartments. For example, the image acquisition device may be installed on the inner wall of the box.

[0198] The image acquisition device is used to acquire the image to be corrected corresponding to the object to be measured.

[0199] The image processing device is the image processing device as described in any of the above embodiments, and is used to process the image to be corrected.

[0200] According to the refrigeration equipment provided in the embodiment of the present application, the image to be corrected is segmented by using the corner point information of the middle chamber of the image to be corrected, and then the first image obtained by the segmentation is corrected separately, that is, the image to be corrected is corrected multiple times to obtain the target detection image, which effectively reduces the error and information loss that may be caused by one correction, so that a complete image without distortion can be obtained even when the shooting angle is large or the distortion is severe, thereby improving the precision and accuracy of object detection and recognition based on the target detection image.

[0201] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0202] On the other hand, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to perform the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0203] On the other hand, an embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0204] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0205] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0206] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be corrected corresponding to the object to be measured, wherein the image to be corrected includes information of multiple corner points corresponding to multiple compartments in the refrigeration equipment, and the compartments are used to store the object to be measured; Based on at least one corner point information among the multiple corner point information corresponding to each of the compartments, segment the image to be corrected to obtain at least one first image corresponding to the image to be corrected; Correction processing is performed on each of the first images to obtain a target detection image corresponding to the object to be detected.

2. The image processing method according to claim 1, characterized in that: The step of respectively performing correction processing on each of the first images to obtain a target detection image corresponding to the object to be detected includes: Based on at least one corner point information in each of the first images, respectively perform correction processing on each of the first images to obtain a second image corresponding to each of the first images; Perform stitching processing on at least one of the second images to obtain the target detection image.

3. The image processing method according to claim 2, characterized in that: The correcting each of the first images based on at least one corner point information in each of the first images to obtain a second image corresponding to each of the first images includes: Based on at least one corner point information in each of the first images and the to-be-aligned corner point information corresponding to each of the corner point information, acquiring a perspective transformation matrix between the first image and a second image corresponding to the first image; Based on the perspective transformation matrix, perspective transformation is performed on the first image to obtain a second image corresponding to the first image.

4. The image processing method according to claim 3, characterized in that: The step of performing perspective transformation on the first image based on the perspective transformation matrix to obtain a second image corresponding to the first image includes: Based on the perspective transformation matrix, the first image at the first viewing angle is perspective transformed to a second viewing angle to obtain a second image corresponding to the first image.

5. The image processing method according to claim 2, characterized in that: The step of performing stitching processing on at least one of the second images to obtain the target detection image includes: Each of the second images is converted into an image of the same size, and at least one of the converted second images is spliced ​​to obtain the target detection image.

6. The image processing method according to claim 2, characterized in that: The step of performing stitching processing on at least one of the second images to obtain the target detection image includes: Acquire common corner point information between a target second image in at least one of the second images and other second images in at least one of the second images except the target second image; Based on the same corner point information, a first edge to be stitched in the target second image and a second edge to be stitched in the other second image are acquired; all corner point information corresponding to the first edge to be stitched corresponds to all corner point information corresponding to the second edge to be stitched in one-to-one correspondence; The first edge to be spliced ​​and the second edge to be spliced ​​are spliced ​​to obtain the target detection image.

7. The image processing method according to any one of claims 1 to 6, characterized in that: The step of obtaining the image to be corrected corresponding to the object to be measured includes: Acquire at least one of an internal parameter and a distortion parameter corresponding to an image acquisition device used to acquire the image to be corrected; Based on at least one of the internal parameter and the distortion parameter, the initial image corresponding to the object to be measured acquired by the image acquisition device is corrected to obtain the image to be corrected.

8. An image processing device, characterized in that: include: A first processing module is used to obtain an image to be corrected corresponding to the object to be measured, wherein the image to be corrected includes information of multiple corner points corresponding to multiple compartments in the refrigeration equipment, and the compartments are used to store the object to be measured; A second processing module, configured to segment the image to be corrected based on at least one corner point information among the plurality of corner point information corresponding to each of the compartments, to obtain at least one first image corresponding to the image to be corrected; The third processing module is used to perform correction processing on each of the first images respectively to obtain a target detection image corresponding to the object to be detected.

9. A refrigeration device, characterized in that: include: Box; A plurality of compartments, each of which is installed in the box and is used to store objects to be tested; The image processing device as claimed in claim 8, wherein the image processing device is used to process the image to be corrected corresponding to the object to be measured.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the image processing method according to any one of claims 1 to 7 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images