Region perception analysis method and analysis system based on light field image

Abstract

The invention relates to an area perception analysis method and analysis system based on light field images. The method comprises the following steps: obtaining a light field image group of a perception area; determining a feature object in the light field image; after the observation surface is determined, a light field image corresponding to the observation angle is selected and recorded as a reference light field image, and the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within an allowable range; analyzing the surface of the feature object in the reference light field image, and determining a suspected abnormal area; adjusting an observation surface according to the suspected abnormal area; and performing depth estimation on the suspected abnormal regions to obtain a discrete depth distribution diagram, and classifying the suspected abnormal regions according to the discrete depth distribution diagram. According to the region perception analysis method and analysis system based on the light field image, the abnormal region in the transformer substation region can be highly analyzed, the abnormal position of the surface can be found through the analysis mode, and early detection of danger is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a region perception analysis method and analysis system based on light field images. BACKGROUND

[0002] A light field image is an image capable of recording the complete distribution information of light in space. It contains not only the spatial position information (i.e. the brightness or color of the object surface) in the traditional two-dimensional image, but also the propagation direction information (angle dimension) of the light, thereby realizing multi-dimensional capture and reproduction of a three-dimensional scene.

[0003] A significant difference between a light field image and a traditional image is that the traditional image only records two-dimensional spatial information (such as a planar image captured by a CCD / CMOS sensor), while the light field image retains the direction information of the light by increasing the angle dimension, thereby supporting advanced functions such as refocus view transformation and depth estimation.

[0004] For example, for a substation, due to the scale and complexity of the substation, a traditional video monitoring system can only provide a planar image, which has inherent defects (lack of high-level features and limited information) in fault analysis and is difficult to apply to detailed analysis.

[0005] And if a super high-definition monitoring device is used, there are considerable difficulties in device cost and massive data processing, which cannot make up for the data loss problem caused by a single angle. SUMMARY

[0006] The present application provides a region perception analysis method and analysis system based on light field images, which can perform high-level analysis on abnormal regions in a substation region. This analysis method can find abnormal positions on the surface and realize early detection of dangers.

[0007] The above object of the present application is achieved by the following technical solution: In a first aspect, the present application provides a region perception analysis method based on light field images, comprising: obtaining a light field image group of a perception region, the light field image group including multiple light field images; determining a feature object in the light field image; selecting a light field image corresponding to an observation angle after determining an observation surface, denoted as a reference light field image, the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within an allowable range; analyzing the surface of the feature object in the reference light field image to determine a suspected abnormal region; adjusting the observation surface according to the suspected abnormal region so that the projection area of the suspected abnormal region on the observation surface is maximum. performing depth estimation on the suspected abnormal region to obtain a discrete depth distribution map; classifying the suspected abnormal region according to the discrete depth distribution map, the classification including a normal region and an abnormal region.

[0008] In a possible implementation manner of the first aspect, the depth estimation on the suspected abnormal region includes: determining a light field image including the suspected abnormal region, denoted as a reference light field image; sequencing the reference light field image according to the positional relationship, and the area of the suspected abnormal region in the reference light field image tends to decrease in the sequence; selecting interest points in the suspected abnormal region and constructing a height analysis grid using the obtained interest points; associating the height analysis grids in different reference light field images; calculating a pixel value change curve of each interest point in the height analysis grid in the sequence; calculating a change point quantity on the pixel value change curve to obtain a change value; performing depth estimation on the suspected abnormal region using the change value.

[0009] In a possible implementation manner of the first aspect, the depth estimation on the suspected abnormal region using the change value includes: obtaining a change surface according to the change value and determining a peak region and a valley region on the change surface; counting a total quantity of the peak region and the valley region to obtain a quantity total value; using the quantity total value as a depth estimation value.

[0010] In a possible implementation manner of the first aspect, the selecting of the interest points in the suspected abnormal region includes: extracting in the suspected abnormal region using a dynamic threshold to obtain suspected interest points; obtaining angle light information associated with the suspected interest points; counting a dispersion value of the angle light information and screening in the suspected interest points according to the dispersion value of the angle light information to obtain the interest points.

[0011] In a possible implementation manner of the first aspect, the constructing of the height analysis grid using the obtained interest points includes: connecting any two adjacent interest points to obtain a connection line segment; determining a connection line segment having an intersection point; deleting a connection line segment having a longer length from two connection line segments having the intersection point.

[0012] In a possible implementation manner of the first aspect, the associating the height analysis grids in different reference light field images comprises: determining a compression direction of the height analysis grids according to the positional relationship of the reference light field images; selecting a plurality of reference interest points in the height analysis grids in a direction perpendicular to the compression direction, the reference interest points comprising edge reference interest points and non-edge reference interest points; associating the two height analysis grids using the reference interest points.

[0013] In a possible implementation manner of the first aspect, when the two height analysis grids are associated using the reference interest points, the method further comprises: dividing the height analysis grids into regions according to the reference interest points; moving the corresponding reference interest points to coincide; moving the divided interest points so that the two height analysis grids to be associated coincide.

[0014] In a second aspect, the present application provides a region perception analysis device based on light field images, comprising: a data acquisition unit configured to obtain a light field image group of a perception region, the light field image group comprising a plurality of light field images; a feature object determination unit configured to determine a feature object in the light field images; a light field image determination unit configured to select a light field image corresponding to an observation angle after an observation surface is selected, denoted as a reference light field image, the plane where the reference light field image is located being parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface being within an allowable range; a surface analysis unit configured to analyze a surface of the feature object in the reference light field image to determine a suspected abnormal region; an observation surface adjustment unit configured to adjust the observation surface according to the suspected abnormal region so that the projection area of the suspected abnormal region on the observation surface is maximum; a depth estimation unit configured to perform depth estimation on the suspected abnormal region to obtain a discrete depth distribution map; a region classification unit configured to classify the suspected abnormal region according to the discrete depth distribution map, the classification comprising a normal region and an abnormal region.

[0015] In a third aspect, the present application provides a region perception analysis system based on light field images, the system comprising: one or more memories configured to store instructions; and one or more processors configured to call and run the instructions from the memories, and perform the method as described in the first aspect and any possible implementation manner of the first aspect.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, the computer-readable storage medium comprising: The program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0017] In a fifth aspect, the present invention provides a computer program product comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0018] In a sixth aspect, the present invention provides a chip system comprising a processor for implementing the functions involved in the above aspects, such as generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0019] The chip system may be composed of chips, or may include chips and other discrete devices.

[0020] In one possible design, the chip system also includes a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and provided on different devices, connected via wired or wireless means, or the processor and the memory can be coupled on the same device.

[0021] The beneficial effects of adopting the above technical solution are: The light field image-based regional perception analysis method and analysis system disclosed in the present invention can perform height analysis on abnormal areas in the substation area. During the height analysis process, the light and dark changes at different positions are converted into numerical values ​​to replace the height changes. This method can also perform height analysis on small abnormal areas, thereby discovering irregular surfaces and facilitating early detection of dangers. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic flow chart of the steps of a region perception analysis method based on light field images provided by the present invention.

[0023] Figure 2 This is a schematic diagram of a point of interest in a suspected abnormal area provided by the present invention.

[0024] Figure 3 It is a schematic diagram of a height analysis grid provided by the present invention.

[0025] Figure 4 This is a schematic diagram of the principle of using a change value to estimate the depth of a suspected abnormal area provided by the present invention. DETAILED DESCRIPTION

[0026] In order to more clearly understand the technical solutions in the present application, first, the related art is introduced.

[0027] Light field image is based on light field theory, that is, the radiation distribution of light at all positions and directions in space. Light field is the amount of light passing through each point in each direction. From a mathematical point of view, light field is usually represented by a four-dimensional function L(u,v,s,t), where (u,v) and (s,t) represent the coordinates of two parallel planes, and the light passing through the two points can determine its spatial position and direction.

[0028] Light field image needs to sacrifice clarity to some extent, because of the characteristics of multi-dimensional information collection, especially the trade-off between spatial resolution and angular resolution.

[0029] The reasons for sacrificing clarity are as follows: Spatial resolution is divided: in a light field camera using a microlens array, sensor pixels are assigned to multiple microlenses, and the pixel group under each microlens records light rays in different directions, resulting in a limited number of pixels covered by a single microlens; In order to record the direction information of light rays, the sensor needs to allocate part of the pixels to the angle dimension. For example, 100% of the pixels of a traditional camera are used for spatial information, while a light field camera may only use 30% of the pixels to record spatial information, and the rest is used for angle information.

[0030] As can be seen from the above description, light field image has obvious advantages in multi-dimensional information, but there is a certain lack in details, which needs to be compensated by using other technical means.

[0031] The technical solutions in the present application are further described in detail below in combination with the drawings.

[0032] The present application discloses a region perception analysis method based on light field image, please refer to Figure 1 In some examples, the region perception analysis method based on light field image disclosed by the present application includes the following steps: S101, obtaining a light field image group of a perception region, the light field image group including multiple light field images; S102, determining a feature object in the light field image; S103, selecting a light field image corresponding to the observation angle after the observation surface is determined, denoted as a reference light field image, the plane where the reference light field image is located is parallel to the observation surface or the included angle between the plane where the reference light field image is located and the observation surface is within the allowable range; S104, analyzing the surface of the feature object in the reference light field image to determine a suspected abnormal region; S105, adjusting the observation plane according to the suspected abnormal region, so that the projection area of the suspected abnormal region on the observation plane is maximum; S106, performing depth estimation on the suspected abnormal region to obtain a discrete depth distribution map; S107, classifying the suspected abnormal region according to the discrete depth distribution map, the classification including normal region and abnormal region.

[0033] Overall, in step S101, a light field image group of the sensing region is first obtained, and the light field image group includes multiple light field images. Then, in step S102, a feature object in the light field image is determined. The feature object refers to a device or a part of a device, or a region with a problem or a potential problem.

[0034] In step S103, an observation plane is determined, and then light field images corresponding to the observation angle are selected, which are referred to as reference light field images. The plane where the reference light field images are located is parallel to the observation plane, or the angle between the plane where the reference light field images are located and the observation plane is within an allowable range.

[0035] The observation plane refers to a plane that is perpendicular to the line of sight when the feature object is viewed. It is generally required that the projection area of the feature object on the observation plane is maximum. The allowable range of the angle between the plane where the reference light field images are located and the observation plane is generally controlled to be less than or equal to 55°-75°. This is because when the angle is too large, the feature object may not be completely displayed due to occlusion.

[0036] In step S104, the surface of the feature object in the reference light field image is analyzed to determine a suspected abnormal region. The analysis methods used here include edge extraction (to obtain different color regions) and texture analysis (the texture of the suspected abnormal region is different from that of other regions).

[0037] Then, in step S105, the observation plane is adjusted according to the suspected abnormal region, so that the projection area of the suspected abnormal region on the observation plane is maximum. The purpose of this step is to maximize the display of the suspected abnormal region.

[0038] In step S106, depth estimation is performed on the suspected abnormal region to obtain a discrete depth distribution map. Finally, in step S107, the suspected abnormal region is classified according to the discrete depth distribution map, and the classification includes normal region and abnormal region.

[0039] The discrete depth distribution map here refers to the set of heights at different positions on the suspected abnormal area. Through this set, it can be determined whether there is a plane or a height-varying surface on the suspected abnormal area. In the present invention, the suspected abnormal area with a plane is called a normal area, but this does not mean that the area is completely normal. This is because the present invention uses height difference to distinguish between normal and abnormal.

[0040] In some examples, the specific method of depth estimation for suspected abnormal areas is as follows: S201, determining a light field image including a suspected abnormal area, and recording it as a reference light field image; S202, sorting the reference light field images according to the positional relationship, wherein the area of ​​the suspected abnormal region in the reference light field image tends to decrease in the order sequence; S203, selecting points of interest in the suspected abnormal area and constructing a height analysis grid using the obtained points of interest; S204, associating height analysis grids in different reference light field images; S205, calculating a pixel value change curve of each interest point in the height analysis grid in a sequential sequence; S206, calculating the number of change points on the pixel value change curve to obtain a change value; S207: Use the change value to estimate the depth of the suspected abnormal area.

[0041] In steps S201 to S207 , the light field images including the suspected abnormal areas are first determined and the reference light field images are sorted according to the positional relationship. In this case, the areas of the suspected abnormal areas in the reference light field images tend to decrease in the sequential order.

[0042] The positional relationship referred to here can be described as creating multiple guide lines on the suspected abnormal area. There is a moving point on the guide line, and this moving point also exists on a reference light field image. At the same time, a vertical line passing through the moving point and perpendicular to the plane of the reference light field image is required to be perpendicular to the straight line tangent to the guide line.

[0043] This set of reference light field images reflects the sequential changes of the suspected abnormal area under a moving perspective.

[0044] Generally speaking, 3 to 4 sequential sequences are generally selected at this time, that is, three or four angles are selected to observe the suspected abnormal area.

[0045] Then select points of interest in the suspected abnormal area, such as Figure 2 As shown, the obtained interest points are used to construct a height analysis grid, as shown in Figure 3As shown, then the high analysis grid in different reference light field images is associated, and then the pixel value change curve of each interest point in the high analysis grid on the sequential sequence is calculated.

[0046] After obtaining the pixel value change curve, the number of change points on the pixel value change curve is calculated to obtain a change value, and finally the suspected abnormal area is depth estimated using the change value, as shown in Figure 4

[0047] The above method as a whole is to determine the height by using the color change of the suspected abnormal area at different angles. It should be understood that the current method for depth estimation in computer vision mostly uses binocular vision algorithm, but this method is no longer applicable to fine structures, because the error will result in insufficient accuracy of the result, and the volume of the target object is too small to be continuously tracked.

[0048] Using color change can better improve this problem, because if the suspected abnormal area is a plane, the color change on its surface is relatively uniform, but if the suspected abnormal area is a non-plane, the color change on its surface is not uniform, and this non-uniformity is directly related to the height change.

[0049] In some examples, the way of depth estimation of the suspected abnormal area using the change value is as follows: Obtain a change surface according to the change value and determine the peak region and the valley region on the change surface; Statistically obtain a total number of the peak region and the valley region to obtain a total value of the number; Use the total value of the number as the depth estimation value.

[0050] In the above method, the height change is ingeniously converted to a numerical color change for representation. In this method, first, a change surface is obtained according to the change value and the peak region and the valley region on the change surface are determined, then the total number of the peak region and the valley region is statistically obtained to obtain a total value of the number, and at this time the total value of the number is the depth estimation value.

[0051] In some examples, the way of selecting the interest point in the suspected abnormal area is as follows: S301, a dynamic threshold is used to extract in the suspected abnormal area to obtain a suspected interest point; S302, angle light information associated with the suspected interest point is obtained; S303, the dispersion value of the angle light information is statistically obtained and the suspected interest point is screened according to the dispersion value of the angle light information to obtain the interest point.

[0052] ​In steps S301 to S303, the suspected abnormal region is extracted using a dynamic threshold to obtain suspected interest points. The dynamic threshold refers to using different threshold ranges to extract the suspected abnormal region. In each extraction process, some points are obtained. When the area of the points is within the allowed range (generally, the size is required to be greater than 0.2*0.2), the points are all used as suspected interest points.

[0053] Then, the angle light information associated with the suspected interest points is obtained, and the dispersion value of the angle light information is counted and used to screen the suspected interest points to obtain interest points. This way is to screen the suspected interest points by counting the dispersion value of the angle light information, because in step S301, some false suspected interest points may be obtained.

[0054] Here, the false suspected interest points refer to the region corresponding to the suspected interest points being a plane. In the subsequent processing steps, the suspected interest points of the plane type do not have reference value because the color value has a uniform change trend.

[0055] The dispersion value of the angle light information refers to counting the reflected light information. If the region corresponding to the suspected interest points is a plane, the reflected light corresponding to the light from one angle should be parallel, and otherwise should be discrete.

[0056] Here, the dispersion value generally requires that the ratio of the number of non-parallel reflected light to the total number of reflected light is greater than or equal to a set value, which is generally controlled at 0.5-0.8.

[0057] In some examples, the obtained interest points are used to construct a height analysis grid in the following way: Connecting any two adjacent interest points to obtain a connection line segment; Determining the connection line segment with an intersection point; Deleting the connection line segment with a longer length from the two connection line segments with the intersection point.

[0058] The purpose of the above steps is to ensure the consistency of the constructed height analysis grid, which is related later.

[0059] In some examples, the height analysis grids in different reference light field images are associated, including: Determining the compression direction of the height analysis grid according to the positional relationship of the reference light field images; Selecting a plurality of reference interest points in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points including edge reference interest points and non-edge reference interest points; Two height analysis grids are associated using the reference interest points.

[0060] In the above steps, first, the compression direction of the height analysis grid is determined according to the position relationship of the reference light field image, then a plurality of reference interest points are selected in the height analysis grid in the direction perpendicular to the compression direction, and the reference interest points are divided into two categories, namely edge reference interest points (at the edge position of the height analysis grid) and non-edge reference interest points (at the non-edge position of the height analysis grid).

[0061] Finally, the two height analysis grids are associated using the reference interest points, because the reference interest points selected in the direction perpendicular to the compression direction, the relative positions of the reference interest points in the direction perpendicular to the compression direction do not change substantially, the edge reference interest points are used for alignment, one of the height analysis grids is compressed or stretched in the compression direction, and then it is checked whether the non-edge reference interest points are aligned, so that the two height analysis grids are associated using the reference interest points.

[0062] In some possible implementations, when the two height analysis grids are associated using the reference interest points, the following steps are further added: The height analysis grid is regionally divided according to the reference interest points; The corresponding reference interest points are moved to coincide; The divided interest points are moved to make the two height analysis grids to be associated coincide.

[0063] The above method is used to cope with the situation that the two height analysis grids cannot be associated, at this time, the height analysis grid is regionally divided according to the reference interest points, generally, a plurality of straight lines are established in the direction parallel to the compression direction, the straight lines divide the height analysis grid into a plurality of parts, and there is at least one reference interest point between any two straight lines.

[0064] Then, the corresponding reference interest points are moved to coincide, and finally, the divided interest points are moved to make the two height analysis grids to be associated coincide. This way is to tear the height analysis grid by local movement, so as to avoid the problem that the two height analysis grids cannot be coincided due to the whole movement.

[0065] The application further provides a region perception analysis device based on a light field image, comprising: A data acquisition unit is configured to obtain a light field image group of a perception region, and the light field image group comprises a plurality of light field images. A feature object determination unit is configured to determine a feature object in the light field image. The light field image determination unit is configured to determine a light field image corresponding to the observation angle as a reference light field image, and the plane of the reference light field image is parallel to the observation plane or the angle between the plane of the reference light field image and the observation plane is within an allowable range. The surface analysis unit is configured to analyze the surface of the feature object in the reference light field image to determine a suspected abnormal region. The observation plane adjustment unit is configured to adjust the observation plane according to the suspected abnormal region so that the projection area of the suspected abnormal region on the observation plane is maximized. The depth estimation unit is configured to estimate the depth of the suspected abnormal region to obtain a discrete depth distribution map. The region classification unit is configured to classify the suspected abnormal region according to the discrete depth distribution map, and the classification includes a normal region and an abnormal region.

[0066] Further, the depth estimation of the suspected abnormal region includes: determining a light field image including the suspected abnormal region as a reference light field image; sorting the reference light field images according to the positional relationship, and the area of the suspected abnormal region in the reference light field images tends to decrease in the order sequence; selecting interest points in the suspected abnormal region and constructing a height analysis grid using the obtained interest points; associating the height analysis grids in different reference light field images; calculating the pixel value change curve of each interest point in the height analysis grid in the order sequence; calculating the number of change points on the pixel value change curve to obtain a change value; using the change value to estimate the depth of the suspected abnormal region.

[0067] Further, the depth estimation of the suspected abnormal region using the change value includes: obtaining a change surface according to the change value and determining a peak region and a valley region on the change surface; counting the total number of the peak region and the valley region to obtain a total number value; using the total number value as a depth estimation value.

[0068] Further, the selection of the interest points in the suspected abnormal region includes: extracting the suspected interest points in the suspected abnormal region using a dynamic threshold; obtaining angle light information associated with the suspected interest points; counting the dispersion value of the angle light information and screening the suspected interest points according to the dispersion value of the angle light information to obtain the interest points.

[0069] Further, constructing the height analysis grid using the obtained interest points comprises: connecting any two adjacent interest points to obtain a connection line segment; determining the connection line segment with intersection; deleting the connection line segment with longer length from the two connection line segments with intersection.

[0070] Further, correlating the height analysis grids in different reference light field images comprises: determining the compression direction of the height analysis grid according to the position relationship of the reference light field images; selecting a plurality of reference interest points in the height analysis grid in a direction perpendicular to the compression direction, the reference interest points comprising edge reference interest points and non-edge reference interest points; correlating two height analysis grids using the reference interest points.

[0071] Further, when correlating two height analysis grids using the reference interest points, it further comprises: dividing the height analysis grid according to the reference interest points; moving the corresponding reference interest points to coincide; moving the divided interest points to make the two height analysis grids to be correlated coincide.

[0072] In one example, the units in any of the above apparatuses can be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0073] For another example, when the units in the apparatuses can be implemented in the form of a processing element scheduler, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke programs. For another example, these units can be integrated together to be implemented in the form of a system-on-a-chip (SOC).

[0074] In the present application, various objects such as messages / information / devices / network elements / systems / devices / actions / operations / processes / concepts, etc. are named. It should be understood that these specific names do not constitute a limitation on the relevant objects, and the names can be changed according to the scene, context or usage habits, etc. The technical meaning of the technical terms in the present application should be determined mainly from the function and technical effect embodied / executed in the technical scheme.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0076] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0077] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0078] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical scheme. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0079] It should also be understood that in various embodiments of the present application, first, second, etc. are only to represent that a plurality of objects are different. For example, the first time window and the second time window are only to represent different time windows. The above first, second, etc. should not have any effect on the time window itself, and should not limit the embodiments of the present application.

[0080] It should also be understood that, in the various embodiments of the present application, the terms and / or descriptions between different embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0081] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the prior art essentially or the part of the technical solutions can be embodied in the form of a software product, which is stored in a computer readable storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned computer readable storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0082] The present application also provides a region perception analysis system based on light field images, the system comprising: one or more memories for storing instructions; and one or more processors for invoking and running the instructions from the memories, performing the method as described in the foregoing.

[0083] The present application also provides a computer program product comprising instructions which, when executed, cause the terminal device and the network device to perform the operations of the terminal device and the network device corresponding to the above method.

[0084] The present application also provides a chip system comprising a processor for implementing the functions involved in the foregoing, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0085] The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0086] The processor mentioned in any of the above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned feedback information transmission method.

[0087] In one possible design, the chip system further includes a memory configured to store necessary program instructions and data. The processor and the memory can be decoupled and disposed on different devices, and connected through wired or wireless manner to support the chip system to implement various functions in the above-described embodiments. Alternatively, the processor and the memory can be coupled on the same device.

[0088] Optionally, the computer instructions are stored in the memory.

[0089] Optionally, the memory is a storage unit in the chip, such as a register, a cache, etc. The memory can also be a storage unit outside the chip in the terminal, such as a ROM or other type of static storage device that can store static information and instructions, a RAM, etc.

[0090] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0091] The non-volatile memory can be a ROM, a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), or a flash memory.

[0092] The volatile memory can be a RAM used as an external cache. There are various types of RAM, such as a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a direct Rambus dynamic RAM (direct RDRAM).

[0093] The embodiments of the specific implementation are the preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, any equivalent changes made on the structure, shape, and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A region perception analysis method based on light field images, characterized in that: include: Obtaining a light field image group of the sensing area, where the light field image group includes multiple light field images; Determining characteristic objects in a light field image; After determining the observation plane, a light field image corresponding to the observation angle is selected and recorded as a reference light field image, where the plane where the reference light field image is located is parallel to the observation plane or the angle between the plane where the reference light field image is located and the observation plane is within an allowable range; Analyze the surface of the characteristic object in the reference light field image to determine the suspected abnormal area; Adjust the observation surface according to the suspected abnormal area so that the projection area of ​​the suspected abnormal area on the observation surface is maximized; Depth estimation is performed on the suspected abnormal area to obtain a discrete depth distribution map; The suspected abnormal areas are classified according to the discrete depth distribution map, and the classification includes normal areas and abnormal areas.

2. The region perception analysis method based on light field images according to claim 1, characterized in that: Depth estimation of suspected abnormal areas includes: Determine a light field image including the suspected abnormal area, and record it as a reference light field image; The reference light field images are sorted according to their positional relationships. In the sequential order, the area of ​​the suspected abnormal region in the reference light field image tends to decrease. Selecting points of interest in the suspected anomaly area and using the obtained points of interest to construct a height analysis grid; Correlating height analysis grids from different reference light field images; Calculate the pixel value change curve of each interest point in the height analysis grid in the sequential sequence; Calculate the number of change points on the pixel value change curve to obtain the change value; The change value is used to estimate the depth of the suspected abnormal area.

3. The region perception analysis method based on light field images according to claim 2, characterized in that: Using the change value to estimate the depth of the suspected abnormal area includes: Obtain a change surface according to the change value and determine the peak area and the trough area on the change surface; Count the total number of peak areas and trough areas to get the total value; Use the total amount as the depth estimate.

4. The region perception analysis method based on light field images according to claim 2 or 3, characterized in that: Selecting points of interest in suspected abnormal areas includes: Use dynamic threshold to extract suspected abnormal areas and obtain suspected points of interest; Obtaining angular light information associated with suspected points of interest; The discrete values ​​of the angular light information are counted and the suspected points of interest are screened according to the discrete values ​​of the angular light information to obtain the points of interest.

5. The region perception analysis method based on light field images according to claim 4, characterized in that: Using the obtained interest points to construct a height analysis grid includes: Connect any two adjacent points of interest to obtain a connecting line segment; Determine the connecting line segments that have intersections; The longer connecting line segment of the two connecting line segments with an intersection point is deleted.

6. The region perception analysis method based on light field images according to claim 2 or 3, characterized in that: Correlating height analysis grids from different reference light field images involves: Determining the compression direction of the height analysis grid according to the positional relationship of the reference light field image; In a direction perpendicular to the compression direction, a plurality of benchmark interest points are selected in the height analysis grid, where the benchmark interest points include edge benchmark interest points and non-edge benchmark interest points; Relate two height analysis grids using a benchmark point of interest.

7. The region perception analysis method based on light field images according to claim 6, characterized in that: When correlating two height analysis grids using a datum point of interest, also includes: The height analysis grid is divided into regions according to the benchmark points of interest; The corresponding reference points of interest are moved to coincide with each other; Move the divided interest point so that the two height analysis grids being associated coincide.

8. A region perception analysis device based on light field images, characterized in that: include: A data acquisition unit, configured to obtain a light field image group of a sensing area, wherein the light field image group includes a plurality of light field images; a feature object determination unit, configured to determine a feature object in a light field image; a light field image determination unit, configured to determine an observation plane and select a light field image corresponding to an observation angle, which is recorded as a reference light field image, wherein the plane where the reference light field image is located is parallel to the observation plane or the angle between the plane where the reference light field image is located and the observation plane is within an allowable range; a surface analysis unit, configured to analyze the surface of a characteristic object in a reference light field image and determine suspected abnormal areas; An observation surface adjustment unit, used to adjust the observation surface according to the suspected abnormal area so as to maximize the projection area of ​​the suspected abnormal area on the observation surface; A depth estimation unit is used to estimate the depth of the suspected abnormal area and obtain a discrete depth distribution map; The regional classification unit is used to classify suspected abnormal areas according to the discrete depth distribution map, and the classification includes normal areas and abnormal areas.

9. A region perception analysis system based on light field images, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises: The program, when executed by a processor, executes the method according to any one of claims 1 to 7.

Images