Image processing method, storage medium, chip and electronic equipment
By acquiring images in classroom scenes, object detection and spatial transformation matrix are determined, and bird's-eye view is constructed, which solves the problems of high cost and low efficiency of obtaining classroom aerial views in the prior art, and achieves efficient and low-cost bird's-eye view acquisition.
Patent Information
- Application Number
- CN202311297527.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-08
- Publication Date
- 2025-04-08
AI Technical Summary
In classroom scenarios, it is difficult for the prior art to effectively obtain a bird's-eye view of the area to be detected, resulting in high costs and inefficiency.
By acquiring the first image of the area to be detected, the target detection is performed to obtain the position information of the target user, the spatial transformation matrix is determined based on the characteristic information of the bird's eye view and the first image, the coordinate transformation is performed to obtain the position information of the target user in the bird's eye view, and finally the bird's eye view is constructed based on these position information.
Without the need for high-cost operations such as 3D sparse reconstruction, it can effectively reduce the cost of obtaining bird's-eye view of the area to be detected and improve the efficiency of obtaining bird's-eye view.
Smart Images
Figure CN119784828A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, a storage medium, a chip and an electronic device. Background Art
[0002] In classroom lectures, conference rooms and other scenarios, image or video analysis technology can usually be used to analyze teaching activities in the classroom and meeting situations in the conference room to improve the teaching process and meeting process and promote the development of education and meeting processes.
[0003] However, due to the camera perspective imaging, it is difficult to obtain the macroscopic teaching process and spatial distribution information of teaching interaction in the entire classroom space only through intelligent analysis of teaching activities in the image space. For example, due to the imaging perspective, the original rectangular distribution of classrooms and student positions in the real physical space is transformed into a trapezoidal imaging result in the image space, which reflects the characteristics of being larger near and smaller far away. This makes it difficult for intelligent algorithms to intuitively analyze the areas with high and low participation in teaching activities through the image screen.
[0004] Among the existing methods, the method of inverse perspective transformation of image space is mainly concentrated in the field of autonomous driving. In recent years, with the development of autonomous driving technology, more and more autonomous driving solutions adopt the method of inverse perspective transformation of the camera image into the bird's eye view (BEV) space, using the bird's eye view space to eliminate the influence of perspective projection of camera imaging, and then use the panoramic view provided by the bird's eye view for environmental perception, path planning and other functions, which greatly promotes the development of autonomous driving. This kind of method of generating a bird's eye view usually requires the use of the SFM (Structure From Motion) method of a motion camera to perform 3D sparse reconstruction of the space, or use algorithms such as BEVFormer to transform multi-view camera images into a bird's eye view space. However, in a classroom scene, usually only 1 to 2 fixed cameras can be used to collect image data of the classroom scene, which makes the cost of using the above solution to obtain a bird's eye view of the classroom waiting area high. Summary of the invention
[0005] One purpose of the embodiments of the present application is to provide an image processing method, a storage medium, a chip and an electronic device, aiming to improve the existing technical problem of high cost in obtaining a bird's-eye view of an area to be inspected.
[0006] In a first aspect, an embodiment of the present application provides an image processing method, the image processing method comprising:
[0007] Acquire a first image of the area to be detected;
[0008] Performing target detection on the first image to obtain first position information of k target users in the first image;
[0009] Determining a spatial transformation matrix according to the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image;
[0010] Performing coordinate transformation processing on first position information of the k target users in the first image according to the spatial transformation matrix to obtain second position information of the k target users in the bird's-eye view;
[0011] Determine row distribution information and column distribution information of the bird's-eye view according to second position information of k target users in the bird's-eye view;
[0012] An image is constructed according to the row distribution information, the column distribution information and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view.
[0013] In a possible implementation, determining the spatial transformation matrix according to the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image includes:
[0014] Determine four feature point groups according to the image feature information of the first image and the image feature information of the bird's-eye view, wherein the feature point groups include first feature points and second feature points, the first feature points are feature points in the first image, the second feature points are feature points in the first image, and any three first feature points in the four feature point groups are not collinear;
[0015] The spatial transformation matrix is determined by adopting a preset spatial transformation matrix acquisition method according to the four feature point groups.
[0016] In a possible implementation, determining the row distribution information and the column distribution information of the bird's-eye view according to the second position information of the k target users in the bird's-eye view includes:
[0017] Performing column clustering processing according to the second position information of the k target users in the bird's-eye view to obtain column distribution information of the bird's-eye view, wherein the column distribution information includes m columns;
[0018] Acquire the second location information of the target user corresponding to each column in the m columns to obtain m sets of second location information;
[0019] Determine row division information of each column in the m columns according to the number of elements in the second position information set corresponding to each column in the m columns;
[0020] The row distribution information of the bird's-eye view image is determined according to the row division information of each column in the m columns.
[0021] In a possible implementation, performing column clustering processing according to the second position information of the k target users in the bird's-eye view to obtain column distribution information of the bird's-eye view includes:
[0022] Get n clustering information;
[0023] Performing clustering processing on the second location information of the target user in the bird's-eye view according to the n clustering information to obtain n clustering results;
[0024] Obtaining the distribution balance degrees corresponding to the n clustering results respectively to obtain n target distribution balance degrees;
[0025] Determining a target clustering result from the n clustering results according to the n target distribution balance degrees;
[0026] The target clustering result is determined as column distribution information of the bird's-eye view.
[0027] In a possible implementation, the performing image construction according to the row distribution information, the column distribution information, and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view includes:
[0028] Determining a row and column template of the bird's-eye view according to the row distribution information and the column distribution information;
[0029] Determining third position information of the k target users in the row and column template according to the second position information of the k target users in the bird's-eye view and the row and column template;
[0030] The identification information of the k target users is added to the row-column template according to the third position information of the k target users in the row-column template to obtain the bird's-eye view.
[0031] In a possible implementation, determining the third position information of the k target users in the row and column template according to the second position information of the k target users in the bird's-eye view and the row and column template includes:
[0032] Obtaining a first distance between a position indicated by the second position information of a first target user and each template position in the row and column template to obtain a first distance set, wherein the first target user is any one of the k target users;
[0033] Obtaining the minimum distance in the first distance set to obtain a second distance;
[0034] Determining the template position information corresponding to the second distance as the third position information of the first target user;
[0035] Repeat the above method of obtaining the first distance between the second position information of the first target user and each template position in the row and column template to obtain a first distance set, determine the second distance according to the first distance set, and determine the template position information corresponding to the second distance as the third position information of the first target user, until the third position information of k target users in the row and column template is obtained.
[0036] In a possible implementation, the method further includes:
[0037] Acquire first position information of a user to be displayed in a first image;
[0038] Determine the third position information of the user to be displayed from the bird's-eye view according to the first position information;
[0039] The identification information of the user to be displayed is displayed at the position indicated by the third position information of the bird's-eye view.
[0040] In a second aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the image processing method as described in any one of the first aspects.
[0041] In a third aspect, an embodiment of the present application provides a chip, including:
[0042] at least one processor; and,
[0043] a memory communicatively connected to the at least one processor; wherein,
[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image processing method as described in any one of the first aspects.
[0045] In a fourth aspect, an embodiment of the present application provides an electronic device, including:
[0046] Camera;
[0047] at least one processor, in communication with the camera; and
[0048] a memory communicatively connected to the at least one processor; wherein,
[0049] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image processing method as described in any one of the first aspects.
[0050] In the device control method provided in the embodiment of the present application, a first image of the area to be detected is acquired, target detection is performed on the first image to obtain first position information of k target users in the first image, a spatial transformation matrix is determined according to image feature information of the bird's-eye view of the area to be detected and image feature information of the first image, coordinate transformation processing is performed on the first position information of the k target users in the first image according to the spatial transformation matrix to obtain second position information of the k target users in the bird's-eye view, and row distribution information and column distribution information of the bird's-eye view are determined according to the second position information of the k target users in the bird's-eye view. The image is constructed according to the row distribution information, the column distribution information and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view. Therefore, the embodiment of the present application does not need to consume high costs by performing 3D sparse reconstruction and other methods, and can obtain the second position information of the k target users in the bird's-eye view according to the image feature information of the bird's-eye view of the area to be detected and the spatial transformation matrix determined by the image feature information of the first image, and finally construct a bird's-eye view according to the second position information of the k target users in the bird's-eye view, thereby reducing the cost of obtaining the bird's-eye view of the area to be detected and improving the efficiency of obtaining the bird's-eye view. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0052] Figure 1 A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;
[0053] Figure 2A A flowchart of an image processing method provided in an embodiment of the present application;
[0054] Figure 2B A schematic diagram of collecting a first image provided in an embodiment of the present application;
[0055] Figure 2C A schematic diagram of a bird's-eye view provided in an embodiment of the present application;
[0056] Figure 2D A schematic diagram of another bird's-eye view provided in an embodiment of the present application;
[0057] Figure 3 A schematic diagram of the circuit structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0059] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.
[0060] In order to better understand an image processing method provided by an embodiment of the present application, the existing image processing methods are briefly introduced below. In the existing scheme, when obtaining a bird's-eye view of the area to be detected, the area to be detected can be a classroom, a conference room, etc., and it is usually necessary to use the SFM (Structure From Motion) method of a motion camera to perform 3D sparse reconstruction of the space, or use algorithms such as BEVFormer to convert multi-view camera images into bird's-eye views, which makes the cost of obtaining the bird's-eye view high and the efficiency low.
[0061] In order to solve the above-mentioned technical problems, an embodiment of the present application provides an image processing method, which does not require high-cost 3D sparse reconstruction or other methods, and can obtain the second position information of k target users in the bird's-eye view according to the image feature information of the bird's-eye view of the area to be detected and the spatial transformation matrix determined by the image feature information of the first image, and finally construct a bird's-eye view according to the second position information of the k target users in the bird's-eye view, thereby reducing the cost of obtaining the bird's-eye view of the area to be detected and improving the efficiency of obtaining the bird's-eye view.
[0062] The embodiment of the present application provides an image processing system, which includes a camera and an electronic device. The camera and the electronic device can be connected via WIFI, or can be wirelessly connected via other frequency bands such as 2.4 GHz, or can be other wireless connection methods, such as connection based on other IoT wireless protocols. The camera can be a general camera, an industrial camera, etc., and the electronic device can be a computer, a tablet computer, a laptop, a cloud laptop, etc. Of course, the camera and the electronic device can also be connected by wired connection.
[0063] See also Figure 1 , the image processing 100 includes a camera 2 and an electronic device 1. The electronic device 1 includes a first antenna 11, a first processor 12 and a first memory 13, at least one first processor 12 is respectively connected to the first memory 13 in communication, wherein the first memory 13 stores instructions that can be executed by at least one first processor 12, and the instructions are executed by the at least one first processor 12, so that the at least one first processor 12 can perform the image processing method described below. Of course, the camera 2 can also be integrated into the electronic device 1.
[0064] As another aspect of the present application, the present application provides an image processing method, which is applied to an electronic device. Figure 2A , the image processing methods include:
[0065] S201, acquiring a first image of a region to be detected.
[0066] The area to be detected may be a classroom area, a conference room indoor area, etc. The present application embodiment takes a classroom as an example for explanation. When acquiring the first image of the area to be detected, the camera may be set above the classroom blackboard, such as Figure 2B As shown, the camera can continuously record video or take multiple images of the interior area of the classroom, wherein the camera can be an RGB camera, etc. The first image includes an image of the upper body area of the students in the classroom, and the image of the upper body area of the students can be understood as including an image above the shoulders of the students, or at least including a face image of the students, etc.
[0067] S202: Perform target detection on the first image to obtain first position information of k target users in the first image.
[0068] Among them, the YOLOv7 (Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors) algorithm can be used to detect the human body in the first image to obtain the detection frame of the target user. The target user here is a student, that is, the detection frame of the rising. The center coordinates of the detection frame are determined as the first position information of the corresponding student. Of course, other target detection methods can also be used to obtain the first position information of k target users in the first image.
[0069] S203: Determine a spatial transformation matrix according to the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image.
[0070] Before determining the spatial transformation matrix, it can also be determined whether a bird's-eye view of the area to be detected already exists. If a bird's-eye view of the area to be detected already exists, the spatial transformation matrix between the bird's-eye view of the area to be detected and the first image can be directly obtained, and the first position information of the target user in the first image can be converted into the second position information in the bird's-eye view, and the identification information of the target user can be set in the bird's-eye view according to the second position information.
[0071] If there is no bird's-eye view of the area to be detected, it is necessary to obtain the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image. The image feature information may include the specification information of the image. For example, the image feature information of the bird's-eye view includes the specification information of the bird's-eye view. The specification information of the bird's-eye view may be, for example, 700 pixels wide and 900 pixels high. It can be determined based on the actual bird's-eye view requirement information. The image feature information of the first image may include the first location information of the target user, etc.
[0072] When determining the spatial transformation matrix, four feature point groups can be obtained, and the spatial transformation matrix can be determined by using a preset spatial transformation matrix acquisition method based on the four feature point groups. The feature point group includes a first feature point and a second feature point, the first feature point is a feature point in the first image, the second feature point is a feature point in the first image, and any three first feature points in the four feature point groups are not collinear. The preset spatial transformation matrix acquisition method can be an LMS (Least Median of Squares) algorithm or a RANSAC (Random Sample Consensus) algorithm, etc.
[0073] S204: Perform coordinate transformation processing on the first position information of the k target users in the first image according to the space transformation matrix to obtain second position information of the k target users in the bird's-eye view.
[0074] The first position information of the target user in the first image may be multiplied by the spatial transformation matrix to obtain the second position information of the target user in the bird's-eye view. The position information may be represented by position coordinates.
[0075] For example, the second position information may be determined by the method shown in the following formula:
[0076] Q(q1,q2)=H*P(p1,p2),
[0077] Wherein P(p1, p2) represents the first position coordinates (first position information) of a target user in the image space, p1 represents the horizontal coordinate value of point P in the first image space, p2 represents the vertical coordinate value of point P in the first image space, and Q(q1, q2) represents the coordinates of point P in the bird's-eye view space after projection, where q1 represents the horizontal coordinate value of point Q in the bird's-eye view space, and q2 represents the vertical coordinate value of point Q in the bird's-eye view space.
[0078] S205 . Determine row distribution information and column distribution information of the bird's-eye view according to the second position information of the k target users in the bird's-eye view.
[0079] The second location information of the k target users in the bird's-eye view may be subjected to column clustering processing to obtain column distribution information of the bird's-eye view, and then the row distribution information is determined according to the second location information of the target user corresponding to each column in the column distribution information. Thus, the row distribution information and column distribution information of the bird's-eye view may be accurately determined.
[0080] S206 , constructing an image according to the row distribution information, the column distribution information, and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view.
[0081] The row and column template of the bird's-eye view can be determined according to the row distribution information and the column distribution information, and the third position information corresponding to the second position information in the row and column template can be determined according to the row and column template, and the identification information of the corresponding target user can be added to the row and column template according to the third position information to obtain the bird's-eye view. The identification information of the target user can be information used to identify the target user, specifically, it can be a head image of the target user, etc., of course, it can also be a black dot to represent the target user, etc. Figure 2C As shown, Figure 2C A schematic diagram of a bird's eye view is shown. Figure 2C Only the case where the column distribution information includes 3 columns is shown. Of course, it can also include other numbers of columns. This is just an example.
[0082] In this example, a first image of the area to be detected is acquired, target detection is performed on the first image to obtain first position information of k target users in the first image, a spatial transformation matrix is determined according to image feature information of the bird's-eye view of the area to be detected and image feature information of the first image, coordinate transformation processing is performed on the first position information of the k target users in the first image according to the spatial transformation matrix to obtain second position information of the k target users in the bird's-eye view, row distribution information and column distribution information of the bird's-eye view are determined according to the second position information of the k target users in the bird's-eye view, and the spatial transformation matrix is used to transform the first position information of the k target users in the first image according to the spatial transformation matrix. The row distribution information, the column distribution information and the second position information of the k target users in the bird's-eye view are used to construct an image to obtain the bird's-eye view. Therefore, the embodiment of the present application does not need to perform 3D sparse reconstruction and other methods that consume high costs, and can obtain the second position information of the k target users in the bird's-eye view based on the image feature information of the bird's-eye view of the area to be detected and the spatial transformation matrix determined by the image feature information of the first image, and finally construct a bird's-eye view based on the second position information of the k target users in the bird's-eye view, thereby reducing the cost of obtaining the bird's-eye view of the area to be detected and improving the efficiency of obtaining the bird's-eye view.
[0083] In a possible implementation, a method for determining a spatial transformation matrix according to image feature information of a bird's-eye view of the area to be detected and image feature information of the first image includes:
[0084] A1. Determine four feature point groups according to the image feature information of the first image and the image feature information of the bird's-eye view, wherein the feature point groups include first feature points and second feature points, the first feature points are feature points in the first image, the second feature points are feature points in the first image, and any three first feature points in the four feature point groups are not collinear;
[0085] A2. Determine the spatial transformation matrix by using a preset spatial transformation matrix acquisition method according to the four feature point groups.
[0086] The image feature information of the first image may include the coordinate information of the target user in the first image. The image feature information of the bird's-eye view may include the size information of the bird's-eye view.
[0087] The first feature point in the four feature point groups can be, for example, a position point indicated by the first position information of the target user in the upper left corner of the first image, a position point indicated by the first position information of the target user in the upper right corner of the first image, a position point indicated by the first position information of the target user in the lower left corner of the first image, and a position point indicated by the first position information of the target user in the lower right corner of the first image. The second feature point can be the vertex (0, 0) in the upper left corner of the bird's-eye view, the vertex (699, 0) in the upper right corner of the bird's-eye view, the vertex (0, 899) in the lower left corner of the bird's-eye view, and the vertex (699, 899) in the lower right corner of the bird's-eye view. Specifically, it can be as follows Figure 2D As shown, here we take the bird's-eye view with a width of 700 pixels and a height of 900 pixels as an example. Then the points in the upper left corner can be determined as a feature point group, the points in the upper right corner can be determined as a feature point group, the points in the lower left corner can be determined as a feature point group, and the points in the lower right corner can be determined as a feature point group.
[0088] The method for obtaining the first feature point in the upper left corner of the first image can be: S1. Construct a first coordinate set according to the first position information of k target users; S2. Sort the coordinates in the first coordinate set from small to large according to the size of the horizontal coordinate value, take the first 50% of the sets as the new student coordinate set, and update the coordinate set D. If the number of coordinate sets is an odd number, the first 50% are rounded down; S3. If the number of student coordinate sets D is 1 at this time, output the point as the feature point in the upper left corner; otherwise, continue to execute step S4); S4. Sort the student coordinate set of S3 from small to large according to the size of the vertical coordinate value, take the first 50% of the sets as the new student coordinate set, and update the coordinate set D. If the number of coordinate sets is an odd number, the first 50% are rounded down; S5. If the number of student coordinate sets D is 1 at this time, output the point as the feature point in the upper left corner; otherwise, continue to execute step S2). The methods for obtaining the first feature point of the upper right corner, the first feature point of the lower left corner, and the first feature point of the lower right corner can all refer to the aforementioned method for obtaining the first feature point of the upper left corner, which will not be repeated here.
[0089] After determining the four feature point groups, a preset spatial transformation matrix acquisition method is used to determine the spatial transformation matrix. The preset spatial transformation matrix acquisition method may be an LMS (Least Median of Squares) algorithm or a RANSAC (Random Sample Consensus) algorithm.
[0090] In this example, by acquiring four feature point groups and determining a space transformation matrix according to the four feature point groups, the efficiency of acquiring the space transformation matrix is improved.
[0091] In a possible implementation, a method for determining row distribution information and column distribution information of the bird's-eye view according to second position information of k target users in the bird's-eye view includes:
[0092] B1. Performing column clustering processing according to the second position information of the k target users in the bird's-eye view to obtain column distribution information of the bird's-eye view, where the column distribution information includes m columns;
[0093] B2. Acquire the second location information of the target user corresponding to each column in the m columns to obtain m sets of second location information;
[0094] B3. Determine row division information of each column in the m columns according to the number of elements in the second position information set corresponding to each column in the m columns;
[0095] B4. Determine the row distribution information of the bird's-eye view according to the row division information of each column in the m columns.
[0096] N clustering information may be obtained, and column clustering processing may be performed on the second location information of k target users in the bird's-eye view according to the n clustering information to obtain column distribution information of the bird's-eye view. The n clustering information may include information of n cluster clusters, and a cluster cluster may be understood as the number of clustered columns, for example, the number of columns may be 2-8, etc.
[0097] The second position information of the target user corresponding to each of the m columns can be extracted respectively, and can be determined according to the horizontal coordinate in the second position information of the column to obtain m second position information sets. Specifically, the second position information whose horizontal coordinate is within the horizontal coordinate range of the column is determined as an element in a second position information set.
[0098] The number of elements in the second position information set can be compared with a preset number to obtain row division information of the column. The preset number is set by an experience value or historical data. For example, the row division information can be understood as how many target users the row has. For example, to determine whether the position of the target user in a column is 2 people forming a row or 1 person forming a row, the preset number here is 8. If the total number of people in the column exceeds 8, it is considered that 2 people form a row, otherwise it is considered that 1 person forms a row. Thus, the row division information is obtained.
[0099] The number of target users in each row can be determined according to the row division information of each column in the m columns, thereby obtaining the row distribution information of the bird's-eye view. The row distribution information of the bird's-eye view can be understood as the number of target users in each row in the bird's-eye view.
[0100] In this example, by first determining the column distribution information of the bird's-eye view, and then determining the row distribution information of the bird's-eye view according to the column distribution information, the accuracy of determining the row distribution information is improved.
[0101] In a possible implementation, a method for performing column clustering processing according to the second position information of k target users in the bird's-eye view to obtain column distribution information of the bird's-eye view includes:
[0102] C1, obtain n clustering information;
[0103] C2. performing clustering processing on the second location information of the target user in the bird's-eye view according to the n clustering information to obtain n clustering results;
[0104] C3. Obtain the distribution balance degrees corresponding to the n clustering results respectively to obtain n target distribution balance degrees;
[0105] C4. determining a target clustering result from the n clustering results according to the n target distribution balances;
[0106] C5. Determine the target clustering result as the column distribution information of the bird's-eye view.
[0107] The n clustering information may be set by empirical values or historical data. The n clustering information may include information of n cluster clusters, and the cluster cluster may be understood as the number of cluster columns, for example, the number of columns may be 2-8.
[0108] A general clustering method can be used to cluster the second location information of the target user in the bird's-eye view with the n clustering information to obtain n clustering results. After obtaining the clustering results, the distribution balance corresponding to each clustering result can be obtained. The method for obtaining the distribution balance can be: determining the distribution balance by the difference between the maximum number of cluster members in the group and the minimum number of cluster members. The smaller the difference, the more balanced it is, and the better the balance. When determining the target clustering result from the n clustering results according to the n target distribution balances, the clustering result with the smallest balance difference is determined as the target clustering result. If the balance (difference) is equal, the clustering group result with a smaller number of clusters (columns) is taken as the target clustering result.
[0109] The target clustering result has the corresponding number of columns and the second position information of the target user in each column.
[0110] In this example, the second position information of the target user in the bird's-eye view is clustered using n clustering information to obtain n clustering results, and the clustering result corresponding to the minimum distribution balance among the n clustering results is determined as the target clustering result to obtain the column distribution information of the bird's-eye view, thereby improving the accuracy of determining the column distribution information of the bird's-eye view.
[0111] In a possible implementation, a method for constructing an image according to the row distribution information, the column distribution information, and second position information of k target users in the bird's-eye view to obtain the bird's-eye view includes:
[0112] D1. Determine a row and column template of the bird's-eye view according to the row distribution information and the column distribution information;
[0113] D2. determining third position information of the k target users in the row and column template according to the second position information of the k target users in the bird's-eye view and the row and column template;
[0114] D3. Add identification information of the k target users to the row-column template according to the third position information of the k target users in the row-column template to obtain the bird's-eye view.
[0115] The row-column template can be determined according to the column distribution information and the row distribution information. For example, the column parameters in the row-column template can be determined according to the column distribution information, and the row parameters in the row-column template can be determined according to the row distribution information. The column parameters can include the number of columns, and the row parameters include the number of target users in each row corresponding to each column. At the same time, the row-column template also includes template position information. The template position can be understood as the position information of the row-column block determined according to the row distribution information and the column distribution information. Specifically, Figure 2C As shown, Figure 2C A row and column template is shown in FIG. 2 , with multiple template positions.
[0116] The third position information of the target user in the row and column template can be determined based on the distance between the target user's second position information and each template position in the row and column template. For example, the template position corresponding to the minimum value of the distance between the target user's second position information and each template position in the row and column template can be determined as the third position information, so that the third position information can be quickly acquired.
[0117] After obtaining the third location information, the identification information of the target user can be added to the row and column template according to the third location information to obtain a bird's-eye view. Figure 2D As shown, Figure 2D A schematic diagram showing a second position in a bird's-eye view of a target user and a final bird's-eye view is shown.
[0118] In this example, a bird's-eye view is obtained by obtaining the third position information corresponding to each target user in the row and column template and filling in identification information according to the third position information, thereby improving the efficiency of obtaining the bird's-eye view.
[0119] In a possible implementation, a method for determining third position information of k target users in the row and column template according to the second position information of k target users in the bird's-eye view and the row and column template includes:
[0120] E1. Obtaining a first distance between a position indicated by the second position information of a first target user and each template position in the row and column template to obtain a first distance set, wherein the first target user is any one of the k target users;
[0121] E2. Obtain the minimum distance in the first distance set to obtain a second distance;
[0122] E3. Determine the template location information corresponding to the second distance as the third location information of the first target user;
[0123] E4. Repeat the above method of obtaining the first distance between the second position information of the first target user and each template position in the row and column template to obtain a first distance set, determining the second distance according to the first distance set, and determining the template position information corresponding to the second distance as the third position information of the first target user, until the third position information of k target users in the row and column template is obtained.
[0124] The first distance between the position indicated by the second position information and the template position may be acquired through a distance calculation formula to obtain a first distance set.
[0125] Since the second distance is the minimum distance between the position indicated by the second position information and the template position, it can be shown that the template position has a high degree of overlap with the position indicated by the second position information, and the template position information can be determined as the third position information of the first target user.
[0126] In a possible implementation, after the bird's-eye view is determined, the elements in the bird's-eye view may be annotated. For example, if the user in the bird's-eye view changes, the bird's-eye view may be updated and displayed according to the location information of the changed user, as follows:
[0127] F1. Obtaining first position information of a user to be displayed in a first image;
[0128] F2. Determine the third position information of the user to be displayed from the bird's-eye view according to the first position information;
[0129] F3. Display the identification information of the user to be displayed at the position indicated by the third position information of the bird's-eye view.
[0130] The users to be displayed may be users in the originally acquired bird's-eye view, or may be users who newly appear in the area to be detected and need to be displayed, etc., which is not specifically limited here.
[0131] The method for determining the third position information of the to-be-displayed user from the bird's-eye view according to the first position information may refer to the method for determining the third position information in the aforementioned embodiment, which will not be described in detail here.
[0132] In this example, after the bird's-eye view is determined, if the user in the bird's-eye view changes, the bird's-eye view can be updated according to the changed user's location information, thereby improving the flexibility of obtaining the bird's-eye view and the practicality of the bird's-eye view.
[0133] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.
[0134] See also Figure 3 , Figure 3 The following is a schematic diagram of the circuit structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 300 includes a camera 301, one or more processors 302, and a memory 303. The processor 302 and the memory 303 are communicatively connected, and the camera 301 is connected to the processor 302.
[0135] The memory 303 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the image processing method in the embodiment of the present application. The processor 302 executes the functions of the image processing method provided by the above method embodiment by running the non-volatile software programs, instructions and modules stored in the memory 303.
[0136] The memory 303 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 303 may optionally include a memory remotely arranged relative to the processor 302, and these remote memories may be connected to the processor 302 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0137] The program instructions / modules are stored in the memory 303, and when executed by the one or more processors 302, the image processing method in any of the above method embodiments is executed.
[0138] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, for example Figure 3 A processor 302 in the embodiment may enable the one or more processors to execute the image processing method in any of the above method embodiments.
[0139] An embodiment of the present application also provides a chip, which includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method in any of the above method embodiments.
[0140] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by an electronic device, the electronic device executes any one of the image processing methods described.
[0141] The above described device or equipment embodiments are merely illustrative, wherein the unit modules described as separate components may or may not be physically separated, and the components displayed as module units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. An image processing method, characterized in that: The image processing method comprises: Acquire a first image of the area to be detected; Performing target detection on the first image to obtain first position information of k target users in the first image; Determining a spatial transformation matrix according to the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image; Performing coordinate transformation processing on first position information of the k target users in the first image according to the spatial transformation matrix to obtain second position information of the k target users in the bird's-eye view; Determine row distribution information and column distribution information of the bird's-eye view according to second position information of k target users in the bird's-eye view; An image is constructed according to the row distribution information, the column distribution information and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view.
2. The image processing method according to claim 1, characterized in that: The determining of the spatial transformation matrix according to the image feature information of the bird's-eye view of the area to be detected and the image feature information of the first image comprises: Determine four feature point groups according to the image feature information of the first image and the image feature information of the bird's-eye view, wherein the feature point groups include first feature points and second feature points, the first feature points are feature points in the first image, the second feature points are feature points in the first image, and any three first feature points in the four feature point groups are not collinear; The spatial transformation matrix is determined by adopting a preset spatial transformation matrix acquisition method according to the four feature point groups.
3. The image processing method according to claim 2, characterized in that: The determining the row distribution information and the column distribution information of the bird's-eye view according to the second position information of the k target users in the bird's-eye view includes: Performing column clustering processing according to the second position information of the k target users in the bird's-eye view to obtain column distribution information of the bird's-eye view, wherein the column distribution information includes m columns; Acquire the second location information of the target user corresponding to each column in the m columns to obtain m sets of second location information; Determine row division information of each column in the m columns according to the number of elements in the second position information set corresponding to each column in the m columns; The row distribution information of the bird's-eye view image is determined according to the row division information of each column in the m columns.
4. The image processing method according to claim 1, characterized in that: The performing column clustering processing according to the second position information of the k target users in the bird's-eye view to obtain the column distribution information of the bird's-eye view includes: Get n clustering information; Performing clustering processing on the second location information of the target user in the bird's-eye view according to the n clustering information to obtain n clustering results; Obtaining the distribution balance degrees corresponding to the n clustering results respectively to obtain n target distribution balance degrees; Determining a target clustering result from the n clustering results according to the n target distribution balance degrees; The target clustering result is determined as column distribution information of the bird's-eye view.
5. The image processing method according to any one of claims 1 to 4, characterized in that: The performing image construction according to the row distribution information, the column distribution information and the second position information of the k target users in the bird's-eye view to obtain the bird's-eye view includes: Determining a row and column template of the bird's-eye view according to the row distribution information and the column distribution information; Determining third position information of the k target users in the row and column template according to the second position information of the k target users in the bird's-eye view and the row and column template; The identification information of the k target users is added to the row-column template according to the third position information of the k target users in the row-column template to obtain the bird's-eye view.
6. The image processing method according to claim 5, characterized in that: The determining, based on the second position information of the k target users in the bird's-eye view and the row and column template, the third position information of the k target users in the row and column template comprises: Obtaining a first distance between a position indicated by the second position information of a first target user and each template position in the row and column template to obtain a first distance set, wherein the first target user is any one of the k target users; Obtaining the minimum distance in the first distance set to obtain a second distance; Determining the template position information corresponding to the second distance as the third position information of the first target user; Repeat the above method of obtaining the first distance between the second position information of the first target user and each template position in the row and column template to obtain a first distance set, determine the second distance according to the first distance set, and determine the template position information corresponding to the second distance as the third position information of the first target user, until the third position information of k target users in the row and column template is obtained.
7. The image processing method according to claim 6, characterized in that: The method further comprises: Acquire first position information of a user to be displayed in a first image; Determine the third position information of the user to be displayed from the bird's-eye view according to the first position information; The identification information of the user to be displayed is displayed at the position indicated by the third position information of the bird's-eye view.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the image processing method according to any one of claims 1 to 7.
9. A chip, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: include: Camera; at least one processor, communicatively connected to the camera; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1 to 7.
Citation Information
Cited By
Image processing method and device, equipment and storage medium
CN120544109A