A method and system for region-aware analysis based on light field images

Abstract

The application relates to a light field image-based region perception analysis method and an analysis system, which comprises the following steps: obtaining a light field image group of a perception region; determining a feature object in the light field image; selecting a light field image corresponding to an observation angle after an observation surface is determined, and the light field image is recorded as a reference light field image; the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within an allowable range; the surface of the feature object in the reference light field image is analyzed to determine a suspected abnormal region; the observation surface is adjusted according to the suspected abnormal region; the suspected abnormal region is subjected to depth estimation to obtain a discrete depth distribution diagram, and the suspected abnormal region is classified according to the discrete depth distribution diagram. The light field image-based region perception analysis method and the analysis system can highly analyze abnormal regions in a substation region, the abnormal position on the surface can be found through the analysis method, and early discovery of danger is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a region perception analysis method and analysis system based on light field images. BACKGROUND

[0002] A light field image is an image capable of recording the complete distribution information of light in space. It contains not only the spatial position information (i.e. the brightness or color of the object surface) in the traditional two-dimensional image, but also the propagation direction information (angle dimension) of the light, thereby realizing multi-dimensional capture and reproduction of a three-dimensional scene.

[0003] A significant difference between a light field image and a traditional image is that the traditional image only records two-dimensional spatial information (such as a planar image captured by a CCD / CMOS sensor), while the light field image retains the direction information of the light by increasing the angle dimension, thereby supporting advanced functions such as refocus view transformation and depth estimation.

[0004] For example, for a substation, due to the scale and complexity of the substation, a traditional video monitoring system can only provide a planar image, which has inherent defects (lack of high-level features and limited information) in fault analysis and is difficult to apply to detailed analysis.

[0005] Moreover, if a super high-definition monitoring device is used, there are considerable difficulties in device cost and massive data processing, which cannot make up for the data loss problem caused by a single angle. SUMMARY

[0006] The present application provides a region perception analysis method and analysis system based on light field images, which can perform high-level analysis on abnormal regions in a substation region. This analysis method can find abnormal positions on the surface and realize early detection of dangers.

[0007] The above object of the present application is achieved by the following technical solutions:

[0008] In a first aspect, the present application provides a region perception analysis method based on light field images, comprising:

[0009] Obtaining a light field image group of a perception region, the light field image group including multiple light field images;

[0010] Determining a feature object in the light field image;

[0011] Selecting a light field image corresponding to the observation angle after determining the observation surface, denoted as a reference light field image, the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within an allowable range;

[0012] analyzing a surface of a feature object in a reference light field image to determine a suspected abnormal region;

[0013] adjusting a viewing plane according to the suspected abnormal region to maximize a projected area of the suspected abnormal region on the viewing plane;

[0014] performing depth estimation on the suspected abnormal region to obtain a discrete depth distribution map;

[0015] classifying the suspected abnormal region according to the discrete depth distribution map, the classification including a normal region and an abnormal region.

[0016] In a possible implementation manner of the first aspect, the depth estimation on the suspected abnormal region includes:

[0017] determining a light field image including the suspected abnormal region, denoted as a reference light field image;

[0018] sequentially ordering the reference light field image according to the positional relationship, and the area of the suspected abnormal region in the reference light field image tends to decrease in the sequential sequence;

[0019] selecting interest points in the suspected abnormal region and constructing a height analysis grid using the obtained interest points;

[0020] associating the height analysis grids in different reference light field images;

[0021] calculating a pixel value change curve of each interest point in the height analysis grid in the sequential sequence;

[0022] calculating a number of change points on the pixel value change curve to obtain a change value;

[0023] performing depth estimation on the suspected abnormal region using the change value.

[0024] In a possible implementation manner of the first aspect, the depth estimation on the suspected abnormal region using the change value includes:

[0025] obtaining a change surface according to the change value and determining a peak region and a valley region on the change surface;

[0026] counting a total number of the peak region and the valley region to obtain a total number value;

[0027] using the total number value as a depth estimation value.

[0028] In a possible implementation manner of the first aspect, the selecting of the interest points in the suspected abnormal region includes:

[0029] extracting in the suspected abnormal region using a dynamic threshold to obtain suspected interest points;

[0030] Acquire angle light ray information associated with the suspected interest point;

[0031] Statistically acquire a dispersion value of the angle light ray information, and screen the suspected interest point according to the dispersion value of the angle light ray information to obtain the interest point.

[0032] In a possible implementation manner of the first aspect, constructing the height analysis grid using the obtained interest point comprises:

[0033] Connecting any two adjacent interest points to obtain a connection line segment;

[0034] Determining the connection line segment with the intersection point;

[0035] Deleting the connection line segment with the longer length from the two connection line segments with the intersection point.

[0036] In a possible implementation manner of the first aspect, associating the height analysis grids in different reference light field images comprises:

[0037] Determining a compression direction of the height analysis grid according to the positional relationship of the reference light field images;

[0038] Selecting a plurality of reference interest points in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points comprising edge reference interest points and non-edge reference interest points;

[0039] Associating two height analysis grids using the reference interest points.

[0040] In a possible implementation manner of the first aspect, when the two height analysis grids are associated using the reference interest points, the method further comprises:

[0041] Dividing the height analysis grid into regions according to the reference interest points;

[0042] Moving the corresponding reference interest points to coincide;

[0043] Moving through the divided interest points to make the two height analysis grids associated coincide.

[0044] In a second aspect, the present application provides a region perception analysis device based on a light field image, comprising:

[0045] A data acquisition unit is configured to obtain a light field image group of a perception region, and the light field image group comprises a plurality of light field images;

[0046] A feature object determination unit is configured to determine a feature object in the light field image;

[0047] a light field image determination unit configured to determine a light field image corresponding to the observation angle from the observation surface, denoted as a reference light field image, wherein a plane of the reference light field image is parallel to the observation surface or an angle between the plane of the reference light field image and the observation surface is within an allowable range;

[0048] a surface analysis unit configured to analyze a surface of the feature object in the reference light field image to determine a suspected abnormal region;

[0049] an observation surface adjustment unit configured to adjust the observation surface according to the suspected abnormal region to maximize a projection area of the suspected abnormal region on the observation surface;

[0050] a depth estimation unit configured to estimate a depth of the suspected abnormal region to obtain a discrete depth distribution map;

[0051] a region classification unit configured to classify the suspected abnormal region according to the discrete depth distribution map, wherein the classification includes a normal region and an abnormal region.

[0052] In a third aspect, the present application provides a region perception analysis system based on a light field image, the system comprising:

[0053] one or more memories configured to store instructions; and

[0054] one or more processors configured to invoke and run the instructions from the memories to perform the method as described in the first aspect and any possible implementation of the first aspect.

[0055] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium comprising:

[0056] a program, when the program is run by a processor, the method as described in the first aspect and any possible implementation of the first aspect is performed.

[0057] In a fifth aspect, the present application provides a computer program product comprising program instructions, when the program instructions are run by a computing device, the method as described in the first aspect and any possible implementation of the first aspect is performed.

[0058] In a sixth aspect, the present application provides a chip system, the chip system comprising a processor configured to implement the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0059] The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0060] In a possible design, the chip system further includes a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and arranged on different devices and connected through wired or wireless manner, or the processor and the memory can be coupled on the same device.

[0061] The technical scheme has the beneficial effects that:

[0062] The area perception analysis method and analysis system based on the light field image can highly analyze the abnormal area in the substation area, and in the process of high analysis, the light and dark changes at different positions are converted into numerical values to replace the height changes, so that the small area abnormal area can also be highly analyzed, and thus the irregular surface can be found, which is helpful to realize early discovery of danger. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a step flow diagram of an area perception analysis method based on a light field image provided by the application.

[0064] Figure 2 is a schematic diagram of a suspected abnormal area in which an interest point exists.

[0065] Figure 3 is a schematic diagram of a height analysis grid.

[0066] Figure 4 is a principle schematic diagram of depth estimation of a suspected abnormal area using a change value. DETAILED DESCRIPTION

[0067] In order to more clearly understand the technical solutions in the application, first, the related art is introduced.

[0068] The light field image is based on the light field theory, that is, the radiation distribution of light at all positions and directions in space. The light field is the amount of light passing through each point in each direction. From a mathematical point of view, the light field is usually represented by a four-dimensional function L(u,v,s,t), where (u,v) and (s,t) represent the coordinates of two parallel planes, and the light passing through the two points can determine its spatial position and direction.

[0069] The light field image needs to sacrifice clarity to some extent, because due to the characteristics of multi-dimensional information collection, especially the trade-off between spatial resolution and angular resolution.

[0070] The reasons for sacrificing clarity are as follows:

[0071] Spatial resolution is divided: in light field cameras using microlens arrays, sensor pixels are assigned to multiple microlenses, and the group of pixels under each microlens records light rays in different directions, resulting in a limited number of pixels covered by a single microlens.

[0072] In order to record the direction information of light rays, part of the pixels of the sensor need to be allocated to the angle dimension. For example, 100% of the pixels of a traditional camera are used for spatial information, while a light field camera may only use 30% of the pixels to record spatial information, and the rest are used for angle information.

[0073] From the above description, it can be seen that the light field image has obvious advantages in multi-dimensional information, but there is a certain lack in details, which needs to be made up by using other technical means.

[0074] The technical solutions in the application will be further described in detail below in combination with the drawings.

[0075] The application discloses a region perception analysis method based on a light field image. Figure 1 In some examples, the region perception analysis method based on the light field image disclosed by the application includes the following steps:

[0076] S101, obtaining a light field image group of a perception region, the light field image group including multiple light field images;

[0077] S102, determining a feature object in the light field image;

[0078] S103, selecting a light field image corresponding to an observation angle after determining an observation surface, denoted as a reference light field image, the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within an allowable range;

[0079] S104, analyzing the surface of the feature object in the reference light field image to determine a suspected abnormal region;

[0080] S105, adjusting the observation surface according to the suspected abnormal region, so that the projection area of the suspected abnormal region on the observation surface is maximum;

[0081] S106, performing depth estimation on the suspected abnormal region to obtain a discrete depth distribution map;

[0082] S107, classifying the suspected abnormal region according to the discrete depth distribution map, the classification including a normal region and an abnormal region.

[0083] In step S101, a light field image group is obtained, which includes a plurality of light field images. In step S102, a feature object in the light field images is determined. The feature object refers to a device or a part of a device, or a region with a problem or a potential problem.

[0084] In step S103, an observation plane is determined, and a light field image corresponding to the observation angle is selected as a reference light field image. The plane of the reference light field image is parallel to the observation plane, or the angle between the plane of the reference light field image and the observation plane is within an allowable range.

[0085] The observation plane refers to a plane that is perpendicular to the line of sight when the feature object is viewed. In general, the projection area of the feature object on the observation plane is required to be the largest. The allowable range of the angle between the plane of the reference light field image and the observation plane is generally controlled to be less than or equal to 55°-75°. This is because when the angle is too large, the feature object may not be completely displayed due to occlusion.

[0086] In step S104, the surface of the feature object in the reference light field image is analyzed to determine a suspected abnormal region. The analysis methods used herein include edge extraction (to obtain different color regions) and texture analysis (the texture of the suspected abnormal region is different from that of other regions).

[0087] In step S105, the observation plane is adjusted according to the suspected abnormal region, so that the projection area of the suspected abnormal region on the observation plane is the largest. The purpose of this step is to maximize the display of the suspected abnormal region.

[0088] In step S106, depth estimation is performed on the suspected abnormal region to obtain a discrete depth distribution map. In step S107, the suspected abnormal region is classified according to the discrete depth distribution map, including a normal region and an abnormal region.

[0089] The discrete depth distribution map refers to a set of heights at different positions on the suspected abnormal region. Through the set, it can be determined whether the suspected abnormal region is a flat surface or a surface with uneven heights. In the present application, the suspected abnormal region with a flat surface is classified as a normal region. However, this does not mean that the region is completely normal, because the present application uses height differences to distinguish between normal and abnormal.

[0090] In some examples, the specific method of depth estimation on the suspected abnormal region is as follows:

[0091] S201, a light field image including the suspected abnormal region is determined as a reference light field image.

[0092] S202, the reference light field images are sorted according to the positional relationship, and the area of the suspected abnormal region in the reference light field images tends to decrease in the sequential sequence;

[0093] S203, selecting interest points in the suspected abnormal region and constructing a height analysis grid using the obtained interest points;

[0094] S204, correlating the height analysis grids in different reference light field images;

[0095] S205, calculating the pixel value change curve of each interest point in the height analysis grid in the sequential sequence;

[0096] S206, calculating the number of change points on the pixel value change curve to obtain a change value;

[0097] S207, using the change value to estimate the depth of the suspected abnormal region.

[0098] In steps S201 to S207, the light field image including the suspected abnormal region is first determined, and the reference light field images are sorted according to the positional relationship, at this time, it is required that the area of the suspected abnormal region in the reference light field images tends to decrease in the sequential sequence.

[0099] The positional relationship referred to here can be described as creating a plurality of guide lines on the suspected abnormal region, and there is a moving point on the guide line, and the moving point exists in a reference light field image at the same time, and it is required that a vertical line passing through the moving point and perpendicular to the plane where the reference light field image is located is perpendicular to the tangent line of the guide line.

[0100] This set of reference light field images reflects the sequential change of the suspected abnormal region under a moving view angle.

[0101] Generally, at this time, 3 to 4 sequential sequences are generally selected, that is, three or four angles are selected to observe the suspected abnormal region.

[0102] Then, interest points are selected in the suspected abnormal region, as shown in Figure 2 , and a height analysis grid is constructed using the obtained interest points, as shown in Figure 3 , then the height analysis grids in different reference light field images are correlated, and then the pixel value change curve of each interest point in the height analysis grid in the sequential sequence is calculated.

[0103] After obtaining the pixel value change curve, the number of change points on the pixel value change curve is calculated to obtain a change value, and finally the change value is used to estimate the depth of the suspected abnormal region, as shown in Figure 4 .

[0104] The above method is to determine the height by using the color change of the suspected abnormal region at different angles. It should be understood that the current method for depth estimation in computer vision mostly uses binocular vision algorithm, but this method is no longer applicable to fine structures due to the following reasons: the error will result in insufficient accuracy of the result; the volume of the target object is too small to be continuously tracked, etc.

[0105] The use of color change can better improve this problem, because if the suspected abnormal region is a plane, the color change on its surface is relatively uniform, but if the suspected abnormal region is a non-planar surface, the color change on its surface is not uniform, and this non-uniformity is directly related to the height change.

[0106] In some examples, the method for depth estimation of the suspected abnormal region using the change value is as follows:

[0107] Obtain a change surface according to the change value and determine the peak region and the valley region on the change surface;

[0108] Count the total number of the peak region and the valley region to obtain a total number value;

[0109] Use the total number value as the depth estimation value.

[0110] In the above method, the height change is ingeniously converted to a numerical color change for representation. In this method, first, a change surface is obtained according to the change value and the peak region and the valley region on the change surface are determined, then the total number of the peak region and the valley region is counted to obtain a total number value, and the total number value is the depth estimation value.

[0111] In some examples, the method for selecting the interest point in the suspected abnormal region is as follows:

[0112] S301, extract the suspected interest point in the suspected abnormal region using a dynamic threshold;

[0113] S302, obtain the angle light information associated with the suspected interest point;

[0114] S303, count the dispersion value of the angle light information and select the interest point in the suspected interest point according to the dispersion value of the angle light information.

[0115] In steps S301 to S303, the suspected abnormal region is extracted using a dynamic threshold to obtain suspected interest points. The dynamic threshold refers to using different threshold ranges to extract the suspected abnormal region. In each extraction process, some points are obtained. When the area of the points is within the allowed range (generally, the size is required to be greater than 0.2*0.2), the points are all used as suspected interest points.

[0116] Then, the angle light information associated with the suspected interest points is obtained, and the dispersion value of the angle light information is counted and used to screen the suspected interest points to obtain the interest points. This way is to screen the suspected interest points by counting the dispersion value of the angle light information, because in step S301, some false suspected interest points may be obtained.

[0117] Here, the false suspected interest points refer to the region corresponding to the suspected interest points being a plane. The suspected interest points of the plane type do not have reference value in the subsequent processing steps because the color value of the suspected interest points has a uniform change trend.

[0118] The dispersion value of the angle light information refers to counting the reflected light information. If the region corresponding to the suspected interest points is a plane, the reflected light corresponding to the light incident from one angle should be parallel, and otherwise should be discrete.

[0119] Here, the dispersion value generally requires that the ratio of the number of non-parallel reflected light to the total number of reflected light is greater than or equal to a set value, which is generally controlled at 0.5-0.8.

[0120] In some examples, the obtained interest points are used to construct a height analysis grid in the following way:

[0121] Connecting any two adjacent interest points to obtain a connection line segment;

[0122] Determining the connection line segment with an intersection point;

[0123] Deleting the connection line segment with a longer length from the two connection line segments with the intersection point.

[0124] The purpose of the above steps is to ensure the consistency of the constructed height analysis grid, which is related later.

[0125] In some examples, the height analysis grids in different reference light field images are associated, including:

[0126] Determining the compression direction of the height analysis grid according to the positional relationship of the reference light field images;

[0127] a plurality of reference interest points are selected in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points including edge reference interest points and non-edge reference interest points;

[0128] The two height analysis grids are associated using the reference interest points.

[0129] In the above steps, firstly, the compression direction of the height analysis grid is determined according to the positional relationship of the reference light field images, and then a plurality of reference interest points are selected in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points being divided into two categories, namely edge reference interest points (at the edge positions of the height analysis grid) and non-edge reference interest points (at the non-edge positions of the height analysis grid).

[0130] Finally, the two height analysis grids are associated using the reference interest points, because the reference interest points selected in the direction perpendicular to the compression direction do not change substantially in the relative positions perpendicular to the compression direction, the edge reference interest points are used for alignment, one of the height analysis grids is compressed or stretched in the compression direction, and then it is checked whether the non-edge reference interest points are aligned, so that the two height analysis grids are associated using the reference interest points.

[0131] In some possible implementations, when the two height analysis grids are associated using the reference interest points, the following steps are further added:

[0132] The height analysis grid is regionally divided according to the reference interest points;

[0133] The corresponding reference interest points are moved to coincide;

[0134] The divided interest points are moved to make the two height analysis grids to be associated coincide.

[0135] The above method is used to cope with the situation that the two height analysis grids cannot be associated, at this time, the height analysis grid is regionally divided according to the reference interest points, generally, a plurality of straight lines are established in a direction parallel to the compression direction, the straight lines divide the height analysis grid into a plurality of parts, and there is at least one reference interest point between any two straight lines.

[0136] Then, the corresponding reference interest points are moved to coincide, and finally, the divided interest points are moved to make the two height analysis grids to be associated coincide. This method is to tear the height analysis grid by local movement, so as to avoid the problem that the two height analysis grids cannot coincide caused by overall movement.

[0137] The application further provides a region perception analysis device based on a light field image, comprising:

[0138] a data acquisition unit configured to obtain a light field image group of a sensing area, the light field image group comprising a plurality of light field images;

[0139] a feature object determination unit configured to determine a feature object in the light field image;

[0140] a light field image determination unit configured to determine a light field image corresponding to the observation angle after the observation surface is selected, denoted as a reference light field image, a plane where the reference light field image is located is parallel to the observation surface or an included angle between the plane where the reference light field image is located and the observation surface is within an allowable range;

[0141] a surface analysis unit configured to analyze a surface of the feature object in the reference light field image to determine a suspected abnormal area;

[0142] an observation surface adjustment unit configured to adjust the observation surface according to the suspected abnormal area so that a projection area of the suspected abnormal area on the observation surface is maximum;

[0143] a depth estimation unit configured to perform depth estimation on the suspected abnormal area to obtain a discrete depth distribution map;

[0144] a region classification unit configured to classify the suspected abnormal area according to the discrete depth distribution map, the classification including a normal area and an abnormal area.

[0145] Further, the depth estimation on the suspected abnormal area includes:

[0146] determining a light field image including the suspected abnormal area, denoted as a reference light field image;

[0147] sorting the reference light field image according to the positional relationship, and in the order sequence, an area of the suspected abnormal area in the reference light field image tends to decrease;

[0148] selecting a point of interest in the suspected abnormal area and constructing a height analysis grid using the obtained point of interest;

[0149] correlating the height analysis grids in different reference light field images;

[0150] calculating a pixel value change curve of each point of interest in the height analysis grid in the order sequence;

[0151] calculating a number of change points on the pixel value change curve to obtain a change value;

[0152] using the change value to perform depth estimation on the suspected abnormal area.

[0153] Further, the depth estimation on the suspected abnormal area using the change value includes:

[0154] obtaining a change surface according to the change value and determining a crest area and a trough area on the change surface;

[0155] counting the total number of the wave peak region and the wave valley region to obtain a total number value;

[0156] using the total number value as a depth estimation value.

[0157] Further, the selecting the interest points in the suspected abnormal region comprises:

[0158] extracting the suspected interest points in the suspected abnormal region using a dynamic threshold value;

[0159] obtaining angle light information associated with the suspected interest points;

[0160] counting a dispersion value of the angle light information and screening the suspected interest points according to the dispersion value of the angle light information to obtain the interest points.

[0161] Further, the constructing the height analysis grid using the obtained interest points comprises:

[0162] connecting any two adjacent interest points to obtain a connection line segment;

[0163] determining the connection line segment with an intersection point;

[0164] deleting the connection line segment with a longer length from the two connection line segments with the intersection point.

[0165] Further, the correlating the height analysis grids in different reference light field images comprises:

[0166] determining a compression direction of the height analysis grid according to the positional relationship of the reference light field images;

[0167] selecting a plurality of reference interest points in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points comprising edge reference interest points and non-edge reference interest points;

[0168] correlating two height analysis grids using the reference interest points.

[0169] Further, when correlating the two height analysis grids using the reference interest points, further comprises:

[0170] dividing the height analysis grid into regions according to the reference interest points;

[0171] moving the corresponding reference interest points to coincide;

[0172] moving through the divided interest points to make the two height analysis grids to be correlated coincide.

[0173] In one example, the units in any of the above apparatuses can be one or more integrated circuits, configured to implement one or more of the above methods, e.g., one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0174] For another example, when the units in the apparatuses can be implemented in the form of a processing element scheduler, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke a program. For yet another example, these units can be integrated together, implemented in the form of a system-on-a-chip (SOC).

[0175] In the present disclosure, various objects such as messages / information / devices / network elements / systems / apparatuses / actions / operations / processes / concepts, etc. can be named. It can be understood that these specific names do not constitute a limitation on the related objects, and the names can be changed according to factors such as scenarios, contexts, or usage habits. The technical meaning of the technical terms in the present disclosure should be mainly determined according to the functions and technical effects embodied / implemented in the technical solutions.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0177] In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely schematic, and the division of the units is merely a logical function division. In actual implementation, another division manner can be used, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices, or units, and can be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0179] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0180] It should also be understood that in various embodiments of the present application, first, second, etc. are only to represent that a plurality of objects are different. For example, the first time window and the second time window are only to represent different time windows. The above first, second, etc. should not have any effect on the time window itself, and should not limit the embodiments of the present application.

[0181] It should also be understood that in various embodiments of the present application, the terms and / or descriptions between different embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0182] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product stored in a computer readable storage medium includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned computer readable storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and various program code storage media.

[0183] The present application also provides a region perception analysis system based on light field images, the system comprises:

[0184] One or more memories for storing instructions; and

[0185] one or more processors for invoking and executing the instructions from the memory to perform the methods described above.

[0186] The present application also provides a computer program product including instructions which, when executed, cause the terminal device and the network device to perform the operations of the terminal device and the network device corresponding to the methods described above.

[0187] The present application also provides a chip system including a processor for implementing the functions described above, such as generating, receiving, sending, or processing the data and / or information involved in the methods described above.

[0188] The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0189] The processor mentioned above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the programs of the methods described above.

[0190] In a possible design, the chip system further includes a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and arranged on different devices, and connected through wired or wireless means to support the chip system to implement various functions in the embodiments described above. Alternatively, the processor and the memory can be coupled on the same device.

[0191] Optionally, the computer instructions are stored in the memory.

[0192] Optionally, the memory is a storage unit in the chip, such as a register, a cache, etc. The memory can also be a storage unit in the terminal located outside the chip, such as a ROM or other type of static storage device that can store static information and instructions, a RAM, etc.

[0193] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0194] The non-volatile memory can be a ROM, a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), or a flash memory.

[0195] The volatile memory can be a RAM used as an external cache. RAM has many different types, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and direct Rambus RAM.

[0196] The embodiments of the present disclosure are all the preferred embodiments of the present disclosure, and do not limit the protection scope of the present disclosure, so that: any equivalent changes made according to the structure, shape, principle of the present disclosure should be covered within the protection scope of the present disclosure.

Claims

1. A region-aware analysis method based on light field images, characterized in that, include: A light field image set of the perceived region is obtained, which includes multiple light field images; Identify feature objects in the light field image; After determining the observation plane, select the light field image corresponding to the observation angle and record it as the reference light field image. The plane containing the reference light field image is parallel to the observation plane, or the angle between the plane containing the reference light field image and the observation plane is within the allowable range. The surface of the feature object is analyzed in the reference light field image to identify suspected abnormal areas; Adjust the observation surface according to the suspected abnormal area to maximize the projected area of ​​the suspected abnormal area on the observation surface; Depth estimation is performed on suspected abnormal areas to obtain discrete depth distribution maps; Suspected abnormal areas are classified based on discrete depth distribution maps, including normal areas and abnormal areas. Depth estimation of suspected anomaly areas includes: The light field image that includes the suspected anomalous region is identified and denoted as the reference light field image; The reference light field images are sorted according to their positional relationship. In the sequential sequence, the area of ​​the suspected abnormal region in the reference light field image tends to decrease. Select points of interest in suspected anomaly areas and use the obtained points of interest to construct a height analysis mesh; Correlate the height analysis grids in different reference light field images; Calculate the pixel value variation curve of each point of interest in the height analysis grid in a sequential sequence; Calculate the number of points of change on the pixel value change curve to obtain the change value; Use the change values ​​to estimate the depth of suspected anomaly areas; Using change values ​​to estimate the depth of suspected anomaly regions includes: Based on the change value, the change surface is obtained, and the peak and trough regions on the change surface are determined; The total number of peak and trough regions is counted to obtain the total quantity. Use the total quantity as a depth estimate.

2. The region-aware analysis method based on light field images according to claim 1, characterized in that, Selecting points of interest in suspected anomaly areas includes: Dynamic thresholds are used to extract suspected points of interest in suspected abnormal regions. Obtain angular ray information associated with suspected points of interest; The dispersion values ​​of angular ray information are statistically analyzed, and points of interest are selected from suspected points of interest based on these dispersion values.

3. The region-aware analysis method based on light field images according to claim 2, characterized in that, Constructing a height analysis mesh using the obtained points of interest includes: Connect any two adjacent points of interest to obtain a connecting line segment; Identify the connecting line segments that intersect; Delete the longer connecting line segment among two intersecting line segments.

4. The region-aware analysis method based on light field images according to claim 1, characterized in that, Correlating the height analysis grid in different reference light field images includes: The compression direction of the height analysis grid is determined based on the positional relationship of the reference light field image; In the direction perpendicular to the compression direction, multiple reference interest points are selected in the height analysis grid. The reference interest points include edge reference interest points and non-edge reference interest points. Two height analysis grids are associated using a baseline point of interest.

5. The region-aware analysis method based on light field images according to claim 4, characterized in that, When associating two height analysis grids using a reference point of interest, it also includes: The height analysis grid is divided into regions based on the reference interest points; The corresponding reference point of interest is moved to coincide with the target point of interest. Move the divided points of interest so that the two height analysis grids that are being associated coincide.

6. A region-aware analysis device based on light field images, characterized in that, include: The data acquisition unit is used to obtain a light field image group of the sensing area, which includes multiple light field images; Feature object determination unit, used to determine feature objects in light field image; The light field image determination unit is used to select the light field image corresponding to the observation angle after determining the observation surface, and denoted as the reference light field image. The plane where the reference light field image is located is parallel to the observation surface or the angle between the plane where the reference light field image is located and the observation surface is within the allowable range. The surface analysis unit is used to analyze the surface of the feature object in the reference light field image to identify suspected abnormal areas; The observation surface adjustment unit is used to adjust the observation surface according to the suspected abnormal area, so as to maximize the projected area of ​​the suspected abnormal area on the observation surface. The depth estimation unit is used to estimate the depth of suspected abnormal areas and obtain a discrete depth distribution map. The region classification unit is used to classify suspected abnormal regions based on the discrete depth distribution map. The classification includes normal regions and abnormal regions. Depth estimation of suspected anomaly areas includes: The light field image that includes the suspected anomalous region is identified and denoted as the reference light field image; The reference light field images are sorted according to their positional relationship. In the sequential sequence, the area of ​​the suspected abnormal region in the reference light field image tends to decrease. Select points of interest in suspected anomaly areas and use the obtained points of interest to construct a height analysis mesh; Correlate the height analysis grids in different reference light field images; Calculate the pixel value variation curve of each point of interest in the height analysis grid in a sequential sequence; Calculate the number of points of change on the pixel value change curve to obtain the change value; Use the change values ​​to estimate the depth of suspected anomaly areas; Using change values ​​to estimate the depth of suspected anomaly regions includes: Based on the change value, the change surface is obtained, and the peak and trough regions on the change surface are determined; The total number of peak and trough regions is counted to obtain the total quantity. Use the total quantity as a depth estimate.

7. A region-aware analysis system based on light field images, characterized in that, The system includes: One or more memories for storing instructions; and One or more processors are configured to retrieve and execute the instructions from the memory to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: The program, when run by a processor, executes the method as described in any one of claims 1 to 5.

Images