An image reflection detection method, device and processing equipment

Abstract

The application provides an image reflection detection method, device and processing equipment, which can accurately identify images with reflections. The method comprises: obtaining a first frame image and a second frame image in video data, wherein the first frame image is an image to be detected for reflections, and the second frame image is a frame image adjacent to the first frame image in the video data; identifying a first feature point group of the first frame image and a second feature point group of the second frame image respectively; identifying a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein the matching feature point group comprises two feature points included in the first feature point group and the second feature point group respectively and having a matching relationship; and determining that the first frame image has reflections if the plurality of matching feature point groups comprise at least two feature point groups with different motion directions, wherein the motion direction of the feature point group is determined by the positions of the corresponding two feature points.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of images, in particular to an image reflection detection method and device and processing equipment. BACKGROUND

[0002] In logistics operations, it is often necessary to count the logistics pieces placed on site, which may also involve taking pictures of these logistics pieces. The obtained images can provide data support on images for subsequent work.

[0003] Due to the influence of logistics pieces, logistics equipment and other items on the site during shooting, the shooting space and angle are limited, and some images may have reflection. The image area with reflection means that the image content is not good, the image quality is poor, and it is difficult to identify the logistics pieces in the image. Obviously, the image with reflection has low value.

[0004] In the research of the prior art, the inventors found that although there are reflection detection schemes in the prior art, in some reflection detection scenarios, the detection accuracy is low. SUMMARY

[0005] The present application provides an image reflection detection method, device and processing equipment. By comparing the motion flow of the feature points of adjacent frames of images, the image with reflection can be accurately identified, and the presence of weak reflection in the image can be accurately determined.

[0006] In a first aspect, the present application provides an image reflection detection method, which comprises:

[0007] obtaining a first frame image and a second frame image in video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is an adjacent frame image of the first frame image in the video data;

[0008] identifying a first feature point group of the first frame image and a second feature point group of the second frame image, respectively;

[0009] based on the first feature point group and the second feature point group, identifying a plurality of matching feature point groups, wherein each matching feature point group includes two feature points included in the first feature point group and the second feature point group, respectively, and having a matching relationship;

[0010] if the plurality of matching feature point groups includes at least two feature point groups with different motion flow directions, it is determined that the first frame image has reflection, wherein the motion flow direction of the feature point group is determined by the positions of the corresponding two feature points.

[0011] In a first possible implementation manner of the first aspect of the present application, the second frame image is a thumbnail of the third frame image, and before the second frame image is acquired, the method further includes:

[0012] acquiring the third frame image;

[0013] performing equal-ratio reduction processing on the third frame image according to a preset image size to obtain a thumbnail as the second frame image;

[0014] caching the second frame image.

[0015] In a second possible implementation manner of the first aspect of the present application, before the first feature point group of the first frame image and the second feature point group of the second frame image are respectively identified, the method further includes:

[0016] separately detecting whether the first frame image has a reflection image feature;

[0017] separately detecting whether the second frame image has a reflection image feature;

[0018] if neither of the first frame image and the second frame image has the reflection image feature, triggering the first feature point group and the second feature point group to be respectively identified.

[0019] In a third possible implementation manner of the first aspect of the present application, the detection processing of the reflection image feature is implemented through the following steps:

[0020] extracting a parameter value of an S channel and a parameter value of a V channel of each pixel in a target image, and performing binaryzation processing on each pixel according to a preset S channel parameter threshold and a preset V channel parameter threshold, wherein the target image is the first frame image and / or the second frame image, and the S channel and the V channel are color channels in an HSV color space;

[0021] performing connected processing on the target image after the binaryzation processing to obtain a plurality of connected regions;

[0022] selecting an edge point, a center point and a midpoint of the connected region, wherein the midpoint is a point between the edge point and the center point;

[0023] if the parameter value of the V channel of the midpoint is greater than the parameter value of the V channel of the edge point and less than the parameter value of the V channel of the center point, it is determined that the target image has the reflection image feature.

[0024] In a fourth possible implementation manner of the first aspect of the present application, after the plurality of matching feature point groups are identified based on the first feature point group and the second feature point group, the method further includes:

[0025] If the multiple matched feature point groups only include one feature point group with the same motion flow direction, a third feature point group is identified in the first feature point group, and a fourth feature point group is identified in the second feature point group, wherein the third feature point group is composed of feature points in the first feature point group that are not matched in the matched feature point group, and the fourth feature point group is composed of feature points in the second feature point group that are not matched in the matched feature point group;

[0026] Based on the third feature point group and the fourth feature point group, multiple secondary matched feature point groups are identified, wherein each secondary matched feature point group includes two feature points with a matching relationship in the third feature point group and the fourth feature point group, respectively;

[0027] If the multiple secondary matched feature point groups include at least two secondary feature point groups with different motion flow directions, it is determined that the first frame image has a reflection, wherein the motion flow direction of the secondary feature point group is determined by the positions of the corresponding two feature points.

[0028] In combination with the first aspect of the present application, in a fifth possible implementation manner of the first aspect of the present application, after it is determined that the first frame image has a reflection, the method further includes:

[0029] identifying an image region with the reflection in the first frame image, and outputting the first frame image with the image region with the reflection identified; or

[0030] segmenting an image of the image region with the reflection from the first frame image, and outputting the image of the image region with the reflection.

[0031] In combination with the first aspect of the present application, in a sixth possible implementation manner of the first aspect of the present application, the video data is obtained by shooting at a logistics node, the second frame image is a previous frame image of the first frame image, and the image contents of the first frame image and the second frame image each include a shelf on which a logistics piece is placed.

[0032] In a second aspect, the present application provides an image reflection detection device, which includes:

[0033] a transceiving unit configured to obtain a first frame image and a second frame image from video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is a neighboring frame image of the first frame image in the video data;

[0034] The processing unit is configured to identify a first feature point group of the first frame image and a second feature point group of the second frame image respectively, identify a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein each matching feature point group includes two feature points included in the first feature point group and the second feature point group respectively and having a matching relationship, and determine that the first frame image has a reflection if the plurality of matching feature point groups include at least two feature point groups having different motion directions, wherein the motion direction of each feature point group is determined based on the positions of the two feature points included in the feature point group.

[0035] With reference to the second aspect, in a first possible implementation manner of the second aspect, the transceiver is further configured to:

[0036] obtain a third frame image;

[0037] The processing unit is further configured to:

[0038] perform equal-ratio reduction processing on the third frame image according to a preset image size to obtain a thumbnail image as the second frame image;

[0039] cache the second frame image.

[0040] With reference to the second aspect, in a second possible implementation manner of the second aspect, the processing unit is further configured to:

[0041] individually detect whether the first frame image has a reflection image feature;

[0042] individually detect whether the second frame image has a reflection image feature;

[0043] if neither of the first frame image and the second frame image has the reflection image feature, trigger identification of the first feature point group and the second feature point group respectively.

[0044] With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the processing unit is specifically configured to:

[0045] extract a parameter value of an S channel and a parameter value of a V channel of each pixel in a target image, and perform binaryzation processing on each pixel according to a preset S channel parameter threshold value and a preset V channel parameter threshold value, wherein the target image is the first frame image and / or the second frame image, and the S channel and the V channel are color channels in an HSV color space;

[0046] perform connected processing on the target image after the binaryzation processing to obtain a plurality of connected regions;

[0047] select edge points, center points and midpoint of the connected regions, wherein the midpoint is a point between the edge points and the center points;

[0048] If the parameter value of the V channel of the middle point is greater than the parameter value of the V channel of the edge point and less than the parameter value of the V channel of the center point, it is determined that the target image has a reflection image feature.

[0049] In a fourth possible implementation manner of the second aspect of the present application, the processing unit is further configured to:

[0050] If the multiple matching feature point groups only include one feature point group with the same motion flow direction, a third feature point group is identified in the first feature point group, and a fourth feature point group is identified in the second feature point group, wherein the third feature point group is composed of feature points in the first feature point group that are not matched in the matching feature point groups, and the fourth feature point group is composed of feature points in the second feature point group that are not matched in the matching feature point groups;

[0051] Based on the third feature point group and the fourth feature point group, multiple secondary matching feature point groups are identified, wherein each secondary matching feature point group includes two feature points included in the third feature point group and the fourth feature point group respectively and having a matching relationship;

[0052] If the multiple secondary matching feature point groups include at least two secondary feature point groups with different motion flow directions, it is determined that the first frame image has reflection, wherein the motion flow direction of the secondary feature point group is determined by the positions of the corresponding two feature points.

[0053] In a fifth possible implementation manner of the second aspect of the present application, the processing unit is further configured to:

[0054] identifying an image region with reflection in the first frame image, and outputting the first frame image with the image region with reflection identified by the transceiving unit; or

[0055] segmenting an image of the image region with reflection from the first frame image, and outputting the image of the image region with reflection by the transceiving unit.

[0056] In a sixth possible implementation manner of the second aspect of the present application, the video data is obtained by shooting at a logistics node, the second frame image is a previous frame image of the first frame image, and the image contents of the first frame image and the second frame image each include a shelf on which a logistics piece is placed.

[0057] In a third aspect, the present application further provides a processing device including a processor and a memory, the memory storing a computer program, and the processor executing the steps in any method provided in the first aspect of the present application when invoking the computer program in the memory.

[0058] In a fourth aspect, the present application further provides a computer readable storage medium storing a plurality of instructions, which are adapted to be loaded by a processor to execute the steps in any method provided in the first aspect of the present application.

[0059] From the above, the present application has the following beneficial effects:

[0060] The present application takes the first frame image and the second frame image in the video data as input images, wherein the first frame image is an image to be detected for reflection, and the second frame image is a neighboring frame image of the first frame image in the video data; then, a first feature point group of the first frame image and a second feature point group of the second frame image are respectively identified; then, based on the first feature point group and the second feature point group, a plurality of matching feature point groups are identified, wherein each matching feature point group includes two feature points respectively included in the first feature point group and the second feature point group and having a matching relationship; then, the motion flow direction of each feature point group in the plurality of matching feature point groups is detected, and if the plurality of matching feature point groups include at least two feature point groups having different motion flow directions, it is determined that the first frame image has reflection.

[0061] In this process, the motion flow direction of each feature point group in the front and rear frame images is determined in sequence, and if there is no reflection, the motion flow directions of the feature point groups are consistent as a whole, and if there is reflection, since the light source causing the reflection is fixed in the shooting scene, if the shooting angle is changed or the object in the field of view moves, the reflection area will change obviously, so the motion flow direction of the feature point group in the image area with reflection will be greatly different from that of the feature point group in the image area without reflection. At this time, the input image with reflection can be accurately judged. In this way, compared with the prior art which judges whether there is reflection by detecting whether there is a reflection image feature in a single image, the present application can accurately identify the image with reflection by using dynamic detection and comparing the motion flow directions of the feature points of the adjacent frame images. Even if there is only weak reflection in the input image, it can also accurately judge whether there is weak reflection in the image. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0063] Figure 1Fig. 1 is a schematic diagram of a scene according to an embodiment of the present application;

[0064] Figure 2 Fig. 2 is a schematic diagram of another scene according to an embodiment of the present application;

[0065] Figure 3 Fig. 3 is a schematic diagram of a flow according to an embodiment of the present application;

[0066] Figure 4 Fig. 4 is a schematic diagram of a flow of processing a second frame according to an embodiment of the present application;

[0067] Figure 5 Fig. 5 is a schematic diagram of a flow of independent reflection detection according to an embodiment of the present application;

[0068] Figure 6 Fig. 6 is a schematic diagram of a flow of secondary reflection detection according to an embodiment of the present application;

[0069] Figure 7 Fig. 7 is a schematic diagram of a structure of a reflection detection device according to an embodiment of the present application;

[0070] Figure 8 Fig. 8 is a schematic diagram of a structure of a processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0072] In the following description, specific embodiments of the present application will be described with reference to steps and symbolic representations of operations that are performed by one or more computers. Unless specifically stated otherwise, as is apparent from the following discussions, it is appreciated that throughout the specification, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "displaying," or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical electronic signals.

[0073] The principles of the application are operable with numerous other general purpose or special purpose computing, communications environments or configurations. Examples of well known computing systems, environments, and configurations that can be suitable for use with the application include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microcomputer-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above systems or devices.

[0074] The terms "first", "second", and "third" and the like in the specification are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises", "comprising", "includes", "including", and the like, are inclusive only - they do not exclude the presence of other elements.

[0075] Before introducing the application, the relevant content about the application background is introduced first.

[0076] The image reflection detection method and device and the computer readable storage medium provided by the application can be applied to a processing device, and are used for significantly improving the detection accuracy of weak reflections in an input image.

[0077] In the application, the processing device can be understood as a server device, a physical host, or a user equipment (UE) and the like hardware device with data processing capability. The UE can be specifically a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a personal digital assistant (PDA).

[0078] The logistics piece is an article involved in logistics transportation in the logistics field, for example, a logistics piece in a receiving state, a delivery state, or a transportation state. Correspondingly, the first frame image and the second frame image in the image can both include a shelf on which the logistics piece is placed. The logistics piece can be specifically an express piece.

[0079] Further, in the logistics field, the logistics piece can also extend to articles transported by logistics companies other than traditional logistics businesses. For example, the logistics company can also directly act as a supply chain company to provide supply chain services for the consignor, and provide goods and cargo to the consignor through logistics operations. Correspondingly, the logistics nodes constituting the logistics network can include not only nodes performing traditional logistics businesses, but also nodes involved in these supply chain businesses.

[0080] For example, a logistics company can directly deliver goods related to its business to different types of consignees such as supermarkets, convenience stores, shops, or bars, and in the process, inventory checking of the goods may be involved. When images are collected by photographing as data support for inventory checking, due to the influence of the shooting environment and the shooting space, for example, a relatively smooth glass cabinet, there is a risk of reflection. Therefore, the image reflection detection method provided in the present application can be used to detect weak reflection in the image.

[0081] Next, the image reflection detection method provided in the present application will be introduced.

[0082] First, referring to Figure 1 and Figure 2 respectively showing the scene diagram of the image reflection detection method of the present application, and in combination with Figure 3 showing a flow diagram of the image reflection detection method of the present application, the image reflection detection method can specifically include the following steps.

[0083] Step S301, acquiring a first frame image and a second frame image in video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is a neighboring frame image of the first frame image in the video data;

[0084] Step S302, respectively identifying a first feature point group of the first frame image and a second feature point group of the second frame image;

[0085] Step S303, identifying a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein the matching feature point group includes two feature points respectively included in the first feature point group and the second feature point group and having a matching relationship;

[0086] Step S304, if the plurality of matching feature point groups include at least two feature point groups with different motion flow directions, determining that the first frame image has reflection, wherein the motion flow direction of the feature point group is determined by the positions of the corresponding two feature points.

[0087] In combination with Figure 1 , Figure 2 the scene, Figure 3As can be seen from the embodiments, the present application takes a first frame image and a second frame image in video data as input images, wherein the first frame image is an image to be detected for reflection, and the second frame image is a neighboring frame image of the first frame image in the video data; then a first feature point group of the first frame image and a second feature point group of the second frame image are respectively identified; then based on the first feature point group and the second feature point group, a plurality of matching feature point groups are identified, wherein each matching feature point group includes two feature points respectively included in the first feature point group and the second feature point group and having a matching relationship; then a motion flow direction of each feature point group in the plurality of matching feature point groups is detected, and if at least two feature point groups having different motion flow directions are included in the plurality of matching feature point groups, it is determined that the first frame image has reflection, wherein the motion flow direction of the feature point group is determined by positions of the corresponding two feature points.

[0088] In this process, the motion flow direction of each feature point group in the front and back two frame images is determined in sequence, and if there is no reflection, the motion flow directions of the feature point groups are consistent as a whole, and if there is reflection, since the light source causing the reflection is fixed in the shooting scene, if the shooting angle is changed or the object in the field of view moves, the reflection area will change more obviously, and thus the motion flow direction of the feature point group in the image area having reflection will be greatly different from the motion flow direction of the feature point group in the image area not having reflection. At this time, it can be accurately judged that the input image has reflection. In this way, compared with the prior art of judging whether there is reflection by detecting whether there is a reflection image feature in a single image, the present application can accurately identify the image having reflection by using dynamic detection and comparing the motion flow directions of the feature points of the neighboring frame images, and even if there is only weak reflection in the input image, it can be accurately judged whether there is weak reflection in the image.

[0089] Next, specific embodiments of each step of the image reflection detection method of the present application are described in detail.

[0090] In the present application, the first frame image and the second frame image are neighboring frame images, and in the video data, the first frame image can be a previous frame image, and the second frame image can be a next frame image, or the first frame image can be a next frame image, and the second frame image can be a previous frame image, and the order of the two in the frame images in the video data is not limited.

[0091] The video data can be original video data collected on site, or can be video data obtained after processing, for example, video data obtained by configuring a time axis for several images taken on site, video data obtained by speeding up, slowing down, splicing, deleting, etc. processing original video data collected on site.

[0092] The step S301 can be understood as separately extracting the first frame image and the second frame image from the video data, or can be understood as directly obtaining the video data containing the first frame image and the second frame image, and then directly deriving the first frame image and the second frame image from the video data for subsequent use.

[0093] In actual application, the first frame image and the second frame image can be photographed and saved locally on the processing device, or can be photographed and saved on other devices, and then corresponding retrieval processing is performed when the image reflection detection method is triggered, for example, the first frame image and the second frame image are retrieved from device A, or video data X containing the first frame image and the second frame image is retrieved from device B.

[0094] The first frame image is taken as an input image to be detected for reflection, and the second frame image is taken as an input image to assist in detecting whether the first frame image has reflection. Of course, while the second frame image is used to detect whether the first frame image has reflection, the detection result of the first frame image can also be taken as the detection result of whether the second frame image has reflection.

[0095] In an exemplary implementation, in actual application, to reduce the amount of data processing, the second frame image can be a thumbnail.

[0096] Referring to Figure 4 A flowchart of the processing of the second frame image of the present application is shown. The image reflection detection method of the present application can include the following steps in the preprocessing of the second frame image:

[0097] Step S401, obtaining a third frame image;

[0098] It can be understood that the second frame image obtained by the processing device is obtained by reducing. Correspondingly, the processing device can first obtain a third frame image, and then obtain the second frame image by reducing the third frame image.

[0099] Step S402, performing equal-ratio reduction processing on the third frame image according to a preset image size, and taking a thumbnail obtained by the reduction processing as the second frame image;

[0100] When the third frame image is obtained, equal-ratio reduction processing can be performed according to a preset image size, for example, a preset pixel size of a thumbnail, a storage space size occupied by image data, and the like, so as to achieve standardized processing and facilitate storage and processing.

[0101] For example, the reduced image has a short side of 270 pixels.

[0102] 1) When the height of the image before reduction is greater than the width:

[0103]

[0104] wherein, W new is the width of the reduced image, H new is the height of the reduced image, W old is the width of the original image, H old is the height of the original image.

[0105] 2) when the width of the original image is greater than the height:

[0106]

[0107] wherein, W new is the width of the reduced image, H new is the height of the reduced image, W old is the width of the original image, H old is the height of the original image.

[0108] Step S403, buffering the second frame image.

[0109] After the second frame image is obtained by the reducing processing, it is buffered in the cache file storage space.

[0110] It is easy to understand that the image reflection detection method of the present application can be applied on smart phones, PDAs and other UEs. Not only because the data processing amount of the image reflection detection method of the present application is low, the performance requirement of the processing device is small, and considering that the UE applying the image reflection detection method may itself have shooting conditions and have a camera that can shoot images, the real-time detection of reflection can also be performed during shooting (including continuous shooting or video shooting). During shooting, the current shot image or preview image can be taken as the first frame image, and the last frame image can be taken as the second frame image. In this way, real-time reflection detection can be performed, the user can be prompted in real time whether there is reflection, so that the user can adjust the shooting angle in time, so that a better original image can be obtained on the spot, and the adverse effects on logistics operations caused by poor image quality can be avoided.

[0111] In this process, since the thumbnail reducing processing is introduced, the second frame image obtained by reducing is buffered in the cache file storage space, which can also reduce the size of the memory occupied by the UE. After it is determined whether the current frame image has reflection, the occupied memory of the second frame image can be released, so that the processing fluency of the UE system and the user experience can be guaranteed.

[0112] In another exemplary implementation, the first frame image and the second frame image can be individually subjected to a reflection detection before being subjected to the reflection detection by the image reflection detection method of the present application in combination with the comparison of the motion flow of the feature points of the adjacent frame images, so as to detect the input images with obvious reflection in advance, thereby playing a filtering effect.

[0113] Correspondingly, before the step S302 is performed, the reflection detection of the first frame image and the second frame image can be respectively performed. Specifically, the image reflection detection method of the present application can further include the following steps:

[0114] individually detecting whether the first frame image has a reflection image feature;

[0115] individually detecting whether the second frame image has a reflection image feature;

[0116] if neither of the first frame image and the second frame image has the reflection image feature, triggering the identification of the first feature point group and the second feature point group, respectively.

[0117] It should be understood that the reflection detection of the first frame image and the reflection detection of the second frame image can be performed simultaneously or at different times, and the two are not in conflict with each other and do not affect each other.

[0118] It should be understood that the reflection detection of the first frame image and the reflection detection of the second frame image can be performed simultaneously or at different times, and the two are not in conflict with each other and do not affect each other.

[0119] For example, when the detection of whether the first frame image has a reflection in combination with the second frame image is triggered and the second frame image is called, the reflection detection of the second frame image is performed individually; or, when the second frame image is buffered as mentioned above, the reflection detection is performed individually; or, the second frame image is subjected to the reflection detection individually before being subjected to the reduction processing as mentioned above. The specific detection time can be adjusted according to actual needs, and is not limited herein.

[0120] When it is determined that neither of the first frame image and the second frame image has a reflection image feature, the subsequent identification of the first feature point group and the second feature point group can be performed, and then the weak reflection in the first frame image or the second frame image is identified.

[0121] Subsequently, in the first frame image, feature points in the image can be identified through image recognition. The feature points can be identified according to a preset image feature recognition strategy. For example, color features, texture features, geometric shape features, boundary features, spatial features, and other image features in the first frame image can be identified, and then corresponding feature points are obtained. These feature points can be used as a first feature point group. Similarly, feature points identified in the second frame image are used as a second feature point group. In another exemplary implementation, the present application provides an independent reflection detection scheme for an input image. Referring to Figure 5 A flowchart for independent reflection detection of the present application is shown. The independent reflection detection scheme can include the following steps:

[0122] In step S501, the parameter values of the S channel and the parameter values of the V channel of each pixel in the target image are extracted, and each pixel is binarized according to a preset S channel parameter threshold and a preset V channel parameter threshold. The target image is the first frame image and / or the second frame image, and the S channel and the V channel are color channels in the HSV color space.

[0123] It can be understood that the first frame image or the second frame image can be selected as the target image as the input image of the independent reflection detection scheme of the present application.

[0124] Taking the first frame image as the target image as an example, when the first frame image is an RGB image, it can be converted into an HSV image. The HSV image is a color channel image in the Hue-Saturation-Value (HSV) color space, and the RBG image is a color channel image in the Red-Green-Blue (RGB) color space. When the first frame image is an image in another format, it can also be converted into an HSV format image. Of course, if the first frame image is already an HSV image, no format conversion is needed.

[0125] In the HSV format, each pixel in the image has corresponding H, S, and V channel parameter values. In combination with the preset S channel parameter threshold and the preset V channel parameter threshold, each pixel can be binarized. For example, the pixel points with S channel parameter values exceeding the S channel parameter threshold and V channel parameter values exceeding the V channel parameter threshold are set to 1, and the pixel points that do not simultaneously exceed the S channel parameter threshold and the V channel parameter threshold are set to 0, to realize the binary division of the pixel points.

[0126] if(S(u,v)<T S )&&(V(u,v)>T V )

[0127] I(u,v)=1

[0128] else

[0129] I(u,v)=0;

[0130] wherein, u is the parameter value of S channel, v is the parameter value of V channel, T S is the S channel parameter threshold, T V is the V channel parameter threshold.

[0131] For example, the S channel parameter threshold can be 5, the V channel parameter threshold is 253, if the S channel parameter value of X pixel point is 4 (less than 5), the V channel parameter value is 254 (greater than 253), the X pixel point is set to 1.

[0132] Step S502, performing connected processing on the target image after the binaryzation processing, obtaining a plurality of connected regions;

[0133] After the binaryzation processing, the connected region is identified. Wherein, the connected region can be screened through the area threshold:

[0134] if(Q(i)<T Q )

[0135] Region i is discarded

[0136] else

[0137] Region i is retained;

[0138] Wherein, Q(i) is the area of connected region i, T Q is the area threshold.

[0139] Step S503, selecting the edge point, the center point and the midpoint of the connected region, wherein, the midpoint is the point between the edge point and the center point;

[0140] In the present application, the point on the edge of the connected region can be randomly selected as the edge point, and the point between the edge point and the midpoint is selected as the current midpoint.

[0141] Step S504, if the V channel parameter value of the midpoint is greater than the V channel parameter value of the edge point and less than the V channel parameter value of the center point, it is determined that the target image has the reflection image feature.

[0142] By comparing the V channel parameter values of the three points, when the parameter values of the center point, the midpoint and the edge point are sequentially reduced, it can be confirmed that the corresponding connected region has the situation that the reflection gradually weakens from the center to the edge, that is, the connected region has the reflection:

[0143] for i in Q(i):

[0144] if(Vp_center >V p_middle >V p_edge )

[0145] Reflective

[0146] break;

[0147] Among them, V p_center is the parameter value of the V channel at the center point, V p_middle is the parameter value of the V channel at the midpoint, V p_edge It is the parameter value of the V channel of the edge point.

[0148] Under the above settings, on the basis of the initial verification based on the parameter values ​​of the S channel and the parameter values ​​of the V channel, a secondary verification is performed based on the connected area and area, and then a third verification is performed based on the V channel parameter values ​​of the edge point, center point and midpoint. In this way, the triple verification structure of independent reflection detection in this application is realized, so that the presence of reflection can be detected more accurately in a single input image.

[0149] After obtaining the feature point groups of two adjacent frame images, feature point matching can be performed to match feature points that meet similar image features within a close range in the image into a set of matching feature point groups.

[0150] For each group of matching feature points, its motion direction can be determined (which can also be understood as motion trajectory, motion direction, moving trajectory or moving direction, etc., used to describe the change between the positions of two feature points in the matching feature point group). If there is no reflection, then overall, the motion direction of each feature point group tends to be consistent (within the preset flow direction difference range, the corresponding phase difference angle range or and / or phase difference distance range can be configured for distinction); if there is reflection, since the light source causing the reflection is fixed in the shooting scene, if the shooting angle is changed, or the object in the field of view moves, the reflective area will undergo more obvious changes. Therefore, the motion direction of the feature point group in the image area with reflection will be quite different from the motion direction of the feature point group in the image area without reflection.

[0151] by Figure 2 For example, the motion flow direction of the matching feature point group (aa`) in the non-reflective area corresponds to a smaller displacement, while the motion flow direction of the matching feature point group (bb`) in the reflective area corresponds to a larger displacement, and the difference in the motion flow directions of the two is large, exceeding the preset 5-degree angle difference range. Therefore, it can be confirmed that the matching feature point group (bb`) corresponds to a reflective area and there is reflection.

[0152] It is important to understand that Figure 2Only two groups of matching feature point groups (a-a') and (b-b') related to one commodity are shown in the figure, in actual application, a large number of feature points can be identified in adjacent two frames of images, among which, some feature points can be confirmed to have matching relationship and form corresponding matching feature point groups, while some feature points fail to match corresponding feature points.

[0153] In actual application, the image with reflection usually includes two groups of matching feature point groups with different motion flow directions, and if the on-site light source is complex or the article is complex, it can also include multiple groups of matching feature point groups with different motion flow directions.

[0154] In another exemplary implementation, if the reflection is not detected, i.e., if the multiple matching feature point groups do not include at least two groups of feature point groups with different motion flow directions, the reflection detection accuracy is further ensured, and secondary reflection detection can also be performed.

[0155] Referring to Figure 6 A flowchart of the secondary reflection detection of the present application is shown, and the secondary reflection detection of the present application can be implemented by the following steps:

[0156] In step S601, if the multiple matching feature point groups only include one group of feature point groups with the same motion flow direction, a third feature point group is identified in the first feature point group, and a fourth feature point group is identified in the second feature point group, wherein the third feature point group is composed of feature points in the first feature point group that are not matched in the matching feature point groups, and the fourth feature point group is composed of feature points in the second feature point group that are not matched in the matching feature point groups;

[0157] It can be understood that in the secondary reflection detection, the present application is directed to the feature points ignored in the above primary reflection detection.

[0158] That is, the feature points in the first feature point group that are not matched in the above matching feature point groups are the third feature point group at this time, and the feature points in the second feature point group that are not matched in the above matching feature point groups are the fourth feature point group at this time, and the feature points contained in the third feature point group and the fourth feature point group are re-included in the detection range.

[0159] In step S602, based on the third feature point group and the fourth feature point group, multiple secondary matching feature point groups are identified, wherein each secondary matching feature point group includes two feature points included in the third feature point group and the fourth feature point group respectively and having a matching relationship.

[0160] In step S603, if the multiple secondary matching feature point groups include at least two groups of secondary feature point groups with different motion flow directions, it is determined that the first frame of image has reflection, wherein the motion flow direction of the secondary feature point group is determined by the positions of the corresponding two feature points.

[0161] The description of steps S602 and S603 can refer to the above description Figure 3 The description of steps S303 and S304 in the illustrated embodiment is not repeated here.

[0162] By setting the secondary reflection detection, in the case that the first matching feature point group does not detect the presence of reflection, the detection range is reduced again for the feature points other than the first matching feature point group to continue the secondary matching. Thus, compared with the previous matching processing of the feature points, the current matching processing can be more targeted, and the influence of the feature points in the normal image area can be excluded, and a more accurate secondary matching feature point group can be obtained.

[0163] In this case, the secondary reflection detection is completed based on the secondary matching feature point group, and if the reflection is still not detected, that is, if at least two secondary feature point groups with different motion flow directions are not included in the plurality of secondary matching feature point groups, it can be determined that the first frame image does not have reflection, otherwise it can be confirmed that there is reflection.

[0164] Under this double-layer reflection detection, for images with relatively slight reflection, the secondary matching feature point group with slight motion flow direction can still be detected, and the accurate detection of the existing reflection can be greatly guaranteed.

[0165] In actual application, the above-mentioned feature points can be the corner points referred to in image processing, which can be detected by a corner point detection algorithm such as a Features from Accelerated Segment Test (FAST) algorithm, as feature points. Specifically, filtering processing can be performed first, for example, median filtering is used to remove the influence of image noise, then pixel traversal corner point searching is performed, and finally non-maximum suppression is performed to obtain the screened corner points as output.

[0166] The matching of the feature points, that is, the matching of the matching feature point group and the secondary matching feature point group, can be based on the characteristics of gray invariance, small motion, and spatial consistency to track and match the feature points, and multiple detections are performed to realize iterative optimization, and finally the two matched feature points and their corresponding motion flow directions can be obtained.

[0167] After confirming the presence of reflection, the reflection detection result can also be output.

[0168] Specifically, the output processing of the reflection detection result can include:

[0169] identifying the image area with the presence of reflection in the first frame image, and outputting the first frame image with the image area with the presence of reflection identified; or

[0170] Segment the image region with reflection from the first frame image, and output the image region with reflection.

[0171] Taking the application scenario of real-time reflection detection with the current preview image of the UE as the first frame image and the last frame image as the second frame image as an example, when the reflection is detected, the image region with reflection can be identified in the preview image (the first frame image), and real-time display is performed. In this way, the user can know in real time and intuitively that there is reflection and where the reflection is when shooting an image. The user can shoot an image with better quality by adjusting the shooting angle, turning off the light source causing the reflection, adjusting the illumination range of the light source causing the reflection, shielding the illumination of the light source causing the reflection, and other adjustment measures of the shooting conditions.

[0172] Alternatively, the segmented image region with reflection can be separately displayed in the preview interface to remind the user that there is reflection and where the reflection is. The user can shoot an image with better quality by adjusting the shooting angle, turning off the light source causing the reflection, adjusting the illumination range of the light source causing the reflection, shielding the illumination of the light source causing the reflection, and other adjustment measures of the shooting conditions.

[0173] The specific output mode of the reflection detection result can be adjusted according to actual needs or actual application scenarios. For example, the output modes include outputting text information "there is reflection", outputting voice information "there is reflection", adding a reflection mark for identifying the presence of reflection to the first frame image, outputting the first frame image with the image region with reflection marked, and outputting the image of the image region with reflection. Further, two or more output modes can be combined for common output. The specific output mode is not limited herein.

[0174] To better implement the image reflection detection method provided in the present application, the present application further provides an image reflection detection device.

[0175] Referring to Figure 7 , Figure 7 FIG. 1 is a structural schematic diagram of an image reflection detection device according to the present application. In the present application, the image reflection detection device 700 can specifically include the following structures:

[0176] The transceiver unit 701 is configured to acquire a first frame image and a second frame image from video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is a neighboring frame image of the first frame image in the video data.

[0177] The processing unit 702 is configured to identify a first feature point group of the first frame image and a second feature point group of the second frame image respectively, identify a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein each matching feature point group includes two feature points respectively included in the first feature point group and the second feature point group and having a matching relationship, and determine that the first frame image has a reflection light if the plurality of matching feature point groups include at least two feature point groups having different motion directions, wherein the motion direction of each feature point group is determined based on the positions of the corresponding two feature points.

[0178] In an exemplary implementation, the transceiving unit 701 is further configured to:

[0179] obtain a third frame image;

[0180] The processing unit 702 is further configured to:

[0181] perform equal-ratio reduction processing on the third frame image according to a preset image size to obtain a thumbnail image as the second frame image;

[0182] cache the second frame image.

[0183] In another exemplary implementation, the processing unit 702 is further configured to:

[0184] individually detect whether the first frame image has a reflection light image feature;

[0185] individually detect whether the second frame image has a reflection light image feature;

[0186] trigger identification of the first feature point group and the second feature point group respectively if neither of the first frame image and the second frame image has the reflection light image feature.

[0187] In another exemplary implementation, the processing unit 702 is specifically configured to:

[0188] extract a parameter value of an S channel and a parameter value of a V channel of each pixel in a target image, and perform binaryzation processing on each pixel according to a preset S channel parameter threshold and a preset V channel parameter threshold, wherein the target image is the first frame image and / or the second frame image, and the S channel and the V channel are color channels in an HSV color space;

[0189] perform connected processing on the binaryzation-processed target image to obtain a plurality of connected regions;

[0190] select edge points, center points and midpoint of the connected regions, wherein the midpoint is a point between the edge points and the center points;

[0191] determine that the target image has a reflection light image feature if the parameter value of the V channel of the midpoint is greater than the parameter value of the V channel of the edge point and less than the parameter value of the V channel of the center point.

[0192] In yet another example implementation, the processing unit 702 is further configured to:

[0193] if the plurality of matched feature point groups only includes one group of feature points with the same motion flow direction, identify a third feature point group in the first feature point group and a fourth feature point group in the second feature point group, wherein the third feature point group is composed of feature points in the first feature point group that are not matched in the matched feature point groups, and the fourth feature point group is composed of feature points in the second feature point group that are not matched in the matched feature point groups;

[0194] identify a plurality of secondary matched feature point groups based on the third feature point group and the fourth feature point group, wherein each secondary matched feature point group includes two feature points included in the third feature point group and the fourth feature point group respectively and having a matching relationship;

[0195] if the plurality of secondary matched feature point groups includes at least two secondary feature point groups with different motion flow directions, determine that the first frame image has a reflection, wherein the motion flow direction of the secondary feature point group is determined by the positions of the corresponding two feature points.

[0196] In yet another example implementation, the processing unit 702 is further configured to:

[0197] identify an image region in the first frame image where the reflection exists, and output the first frame image with the image region where the reflection exists identified by the transceiving unit 701; or

[0198] segment an image of the image region where the reflection exists from the first frame image, and output the image of the image region where the reflection exists by the transceiving unit 701.

[0199] In yet another example implementation, the video data is obtained at a logistics node, the second frame image is a previous frame image of the first frame image, and the image contents of the first frame image and the second frame image each include a shelf on which a logistics piece is placed.

[0200] The present application also provides a processing device, which is described in detail below with reference to Figure 8 , Figure 8 A structural schematic diagram of the processing device of the present application is shown, and specifically, the processing device of the present application includes a processor 801, a memory 802, and an input and output device 803. The processor 801 is configured to execute a computer program stored in the memory 802 to implement the functions of the steps of the image reflection detection method in any embodiment of the present application; or the processor 801 is configured to execute a computer program stored in the memory 802 to implement the functions of the units in the embodiment of the present application, such as Figures 1 to 6 corresponding to the steps of the image reflection detection method in any embodiment; or the processor 801 is configured to execute a computer program stored in the memory 802 to implement the functions of the units in the embodiment of the present application, such as Figure 7 corresponding to the steps of the image reflection detection method in any embodiment; or the processor 801 is configured to execute a computer program stored in the memory 802 to implement the functions of the units in the embodiment of the present application, such as Figure 7The hardware structure corresponding to the transceiving unit 701 in the transceiving device 700 is the input and output device 803, the hardware structure corresponding to the processing unit 702 is the processor 801, and the memory 802 is used to store the processor 801 to execute the above-mentioned Figures 1 to 6 The computer program required by the image reflection detection method in any embodiment.

[0201] For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 802 and executed by the processor 801 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device.

[0202] The processing device can include, but is not limited to, the processor 801, the memory 802, the input and output device 803. Those skilled in the art can understand that the schematic diagram is only an example of the processing device, and does not constitute a limitation on the processing device, and can include more or less components than the schematic diagram, or combine certain components, or different components, for example, the processing device can also include a network access device, a bus, etc. The processor 801, the memory 802, the input and output device 803 and the network access device are connected through the bus.

[0203] The processor 801 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the processing device, which connects all parts of the device through various interfaces and lines.

[0204] The memory 802 can be used to store computer programs and / or modules, and the processor 801 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 802, and calling data stored in the memory 802. The memory 802 can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data (such as audio data, video data, etc.) created according to the use of the processing device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0205] When the processor 801 is used to execute the computer programs stored in the memory 802, the following functions can be specifically implemented:

[0206] obtaining a first frame image and a second frame image in video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is a frame image adjacent to the first frame image in the video data;

[0207] respectively identifying a first feature point group of the first frame image and a second feature point group of the second frame image;

[0208] identifying a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein each matching feature point group includes two feature points respectively included in the first feature point group and the second feature point group and having a matching relationship;

[0209] if the plurality of matching feature point groups include at least two feature point groups with different motion directions, determining that the first frame image has reflection, wherein the motion direction of the feature point group is determined by the positions of the corresponding two feature points.

[0210] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the image reflection detection device, the processing device and the corresponding units thereof described above can be referred to as Figures 1 to 6 For the description of the image reflection detection method in any embodiment, details are not repeated here.

[0211] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0212] To this end, the present application provides a computer readable storage medium, wherein a plurality of instructions are stored, the instructions can be loaded by a processor to execute the present application as Figures 1 to 6 The specific operation can be referred to the determination method of the logistics line generation strategy in any embodiment. Figures 1 to 6 The description of the image reflection detection method in any embodiment is not repeated here.

[0213] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0214] The instructions stored in the computer readable storage medium can execute the present application as Figures 1 to 6 The steps in the image reflection detection method in any embodiment can be implemented, and the present application as Figures 1 to 6 The beneficial effects of the image reflection detection method in any embodiment can be achieved, and the specific description is given above, which is not repeated here.

[0215] The image reflection detection method, device, processing equipment and computer readable storage medium provided by the present application are described in detail above, and the specific examples are applied to explain the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method and its core idea; meanwhile, for those skilled in the art, the specific implementation mode and application range can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for detecting image reflection, characterized in that: The method comprises: Acquire a first frame image and a second frame image from the video data, wherein the first frame image is an image to be detected for reflection, and the second frame image is an adjacent frame image of the first frame image in the video data; Respectively identifying a first feature point group of the first frame image and a second feature point group of the second frame image; Based on the first feature point group and the second feature point group, identifying a plurality of matching feature point groups, wherein the matching feature point group includes two feature points respectively contained in the first feature point group and the second feature point group and having a matching relationship; If the multiple matching feature point groups include at least two feature point groups with different motion directions, it is determined that there is reflection in the first frame image, wherein the motion direction of the feature point group is determined by the positions of the corresponding two feature points, and wherein the motion direction is used to describe the change between the positions of the two feature points in the matching feature point group.

2. The method according to claim 1, characterized in that The second frame image is a thumbnail of the third frame image. Before acquiring the second frame image, the method further includes: Acquire the third frame image; Reducing the third frame image in proportion to a preset image size, and obtaining a thumbnail thereof as the second frame image; The second frame image is cached.

3. The method according to claim 1, characterized in that Before respectively identifying the first feature point group of the first frame image and the second feature point group of the second frame image, the method further includes: detecting separately whether the first frame of image has a reflective image feature; detecting separately whether the second frame image contains the reflective image feature; If the reflective image features do not exist, the first feature point group and the second feature point group are triggered to be identified respectively.

4. The method according to claim 3, characterized in that The detection and processing of the reflective image features is achieved through the following steps: Extracting the parameter value of the S channel and the parameter value of the V channel of each pixel in the target image, and performing binarization processing on each pixel according to a preset S channel parameter threshold and a preset V channel parameter threshold, wherein the target image is the first frame image and / or the second frame image, and the S channel and the V channel are color channels in the HSV color space respectively; Performing a connection process on the target image after the binarization process to obtain a plurality of connected regions; Selecting an edge point, a center point, and a midpoint of the connected area, wherein the midpoint is a point located between an edge point and the center point; If the parameter value of the V channel of the midpoint is greater than the parameter value of the V channel of the edge point and less than the parameter value of the V channel of the center point, it is determined that the target image has the reflective image feature.

5. The method according to claim 1, wherein After identifying a plurality of matching feature point groups based on the first feature point group and the second feature point group, the method further includes: If the plurality of matching feature point groups include only one feature point group with an identical motion flow direction, a third feature point group is identified in the first feature point group, and a fourth feature point group is identified in the second feature point group, wherein the third feature point group is composed of feature points in the first feature point group that are not matched in the matching feature point group, and the fourth feature point group is composed of feature points in the second feature point group that are not matched in the matching feature point group; Based on the third feature point group and the fourth feature point group, identifying a plurality of secondary matching feature point groups, wherein the secondary matching feature point group includes two feature points respectively included in the third feature point group and the fourth feature point group and having a matching relationship; If the plurality of secondary matching feature point groups include at least two secondary feature point groups with different motion directions, it is determined that the first frame image has reflections, wherein the motion directions of the secondary feature point groups are determined by the positions of the corresponding two feature points.

6. The method according to claim 1, characterized in that After determining that the first frame image has reflection, the method further includes: Identifying an image region where reflection exists in the first frame image, and outputting the first frame image with the image region where reflection exists identified; or, An image of the image area where the reflection exists is segmented from the first frame image, and the image of the image area where the reflection exists is output.

7. The method according to claim 1, characterized in that The video data is captured at a logistics node, the second frame image is the previous frame image of the first frame image, and the image content of the first frame image and the second frame image respectively include shelves on which logistics items are placed.

8. An image reflection detection device, characterized in that: The device comprises: a transceiver unit, configured to obtain a first frame image and a second frame image from the video data, wherein the first frame image is an image of the reflection to be detected, and the second frame image is an adjacent frame image of the first frame image in the video data; A processing unit is used to respectively identify a first feature point group of the first frame image and a second feature point group of the second frame image; based on the first feature point group and the second feature point group, identify multiple matching feature point groups, wherein the matching feature point group includes two feature points that are respectively included in the first feature point group and the second feature point group and have a matching relationship; if the multiple matching feature point groups include at least two groups of feature point groups with different motion flow directions, it is determined that the first frame image has reflections, wherein the motion flow direction of the feature point group is determined by the positions of the corresponding two feature points, wherein the motion flow direction is used to describe the change between the positions of two feature points in the matching feature point group.

9. A processing device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.

Images