Data processing method, thermodynamic diagram generation method and device

By receiving the physical coordinates of the image acquisition device and the calibration coordinates of the terminal, dividing and fusion processing, a heat map adapted to different page resolutions is generated, which solves the problem of thermal map in the prior art and achieves the accuracy of thermal maps and adaptability.

CN119068070APending Publication Date: 2024-12-03BOE TECHNOLOGY GROUP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310639415.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to effectively generate accurate thermal maps, especially at different page resolutions of the scene view, where offsets of hot spot data and noise area interference lead to inaccuracy of thermal maps.

Method used

By receiving the physical coordinates of the image acquisition device and the calibration coordinates of the terminal, dividing the scene coordinate system, fusing the physical coordinates, and generating initial hotspot data based on the calibration coordinates and actual width and height. The terminal then converts to adapt to the current page resolution to generate the target Hot spot data to generate heat maps.

Benefits of technology

The accuracy and adaptability of the heat map are achieved, noise area interference is avoided, and the accurate display of the heat map at different resolutions is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068070A_ABST
    Figure CN119068070A_ABST
Patent Text Reader

Abstract

The invention provides a data processing method and a thermodynamic diagram generation method and device. The data processing method comprises the following steps: receiving a first physical coordinate of at least one object in an image from an image acquisition device under a scene coordinate system and a calibration coordinate for a scene view from a terminal; the calibration coordinate represents the size of a view area corresponding to a scene in the scene view; dividing the scene in the scene coordinate system to obtain a plurality of grids; for each grid in the plurality of grids, based on the first physical coordinates of the at least one object in the scene coordinate system, distributing the first physical coordinates of the at least one object to the corresponding grid; performing fusion processing on the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; generating initial hotspot data based on an image coordinate system according to the calibration coordinates for the scene view, the actual width and the actual height of the scene and the fused physical coordinates; and sending the initial hotspot data to the terminal, so that the terminal generates a thermodynamic diagram in the scene based on the initial hotspot data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method and device, a heat map generation method and device, an image processing method and device, an electronic device, a storage medium, and a computer program product. Background Art

[0002] Heat maps can highlight the degree of interest of objects in various areas of the scene, and can clearly and intuitively display the characteristics of each area. In scenarios such as smart cities and smart retail, heat maps can be used to indicate the degree of interest of objects in various areas, thus providing a basis for adjusting the layout of the scene. Summary of the invention

[0003] The present disclosure provides a data processing method and device, a heat map generation method and device, an image processing method and device, an electronic device, a storage medium, and a computer program product.

[0004] According to one aspect of the present disclosure, a data processing method is provided, including: receiving the first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and the calibrated coordinates for a scene view from a terminal; the calibrated coordinates characterize the size of a view area corresponding to the scene in the scene view; dividing the scene in the scene coordinate system to obtain a plurality of grids; for each of the plurality of grids, based on the first physical coordinates of at least one object in the scene coordinate system, respectively allocating the first physical coordinates of at least one object to the corresponding grid; fusing the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; generating initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene, and the fused physical coordinates; and sending the initial hotspot data to the terminal so that the terminal generates a thermal map within the scene based on the initial hotspot data.

[0005] According to an embodiment of the present disclosure, generating initial hotspot data based on an image coordinate system according to the calibrated coordinates for a scene view, the actual width and actual height of the scene, and the fused physical coordinates includes: converting the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene; determining an initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and actual height of the scene; and determining the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0006] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in an image in a scene coordinate system include the first physical coordinates of the object contained in images respectively captured at multiple image capture moments in the scene coordinate system; based on the first physical coordinates of at least one object in the scene coordinate system, respectively allocating the first physical coordinates of at least one object to corresponding grids includes: determining the number of first physical coordinates of the object contained in images respectively captured at multiple image capture moments; and in response to determining that the number exceeds a preset threshold, allocating the first physical coordinates of the object contained in the image captured at each image capture moment to the corresponding grid, so as to obtain the first physical coordinates of the object contained in each grid at each image capture moment.

[0007] According to an embodiment of the present disclosure, the first physical coordinates of the objects contained in each grid are fused to obtain the fused physical coordinates, including: fusing the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and fusing the initial fused physical coordinates in each grid at multiple image acquisition moments to obtain the fused physical coordinates.

[0008] According to another aspect of the present disclosure, a heat map generation method is provided, including: sending calibrated coordinates for a scene view to a server; the calibrated coordinates represent the size of a view area corresponding to a scene in the scene view; in response to receiving initial hotspot data from the server, obtaining a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; based on the first height, the first width and the original height and the original width, converting the initial hotspot data into target hotspot data at the first page resolution; and generating a heat map within the scene based on the target hotspot data.

[0009] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; based on the first height, the first width and the original height and the original width, converting the initial hotspot data into target hotspot data at the first page resolution includes: determining a first width ratio based on the first width and the original width; determining a first height ratio based on the first height and the original height; converting the initial fused pixel coordinates into target pixel coordinates according to the first width ratio and the first height ratio; converting the initial display radius into a target display radius according to the first width ratio; and determining the target hotspot data according to the target pixel coordinates and the target display radius.

[0010] According to an embodiment of the present disclosure, the above-mentioned heat map generation method also includes: obtaining a second height and a second width of the scene view at a second page resolution; receiving configuration information of the origin coordinates of the scene view; based on the origin coordinates of the scene view, using the second height and the second width as well as the original height and the original width, calibrating the view area in the scene view to obtain calibrated coordinates for the scene view.

[0011] According to an embodiment of the present disclosure, the view area in the scene view is calibrated using the second height and the second width as well as the original height and the original width to obtain the calibrated coordinates for the scene view, including: receiving a calibration operation for the view area to determine calibrated position information for the view area; determining a second height ratio based on the second height and the original height; determining a second width ratio based on the second width and the original width; and determining the calibration coordinates based on the second height ratio, the second width ratio, and the calibrated position information for the view area.

[0012] According to another aspect of the present disclosure, there is provided an image processing method, comprising: acquiring an image captured at each image capture moment; determining the pixel coordinates of at least one object in the image in an image coordinate system; converting the pixel coordinates of at least one object in the image coordinate system into a second physical coordinate in a scene coordinate system relative to the origin coordinates of the image coordinate system; correcting the second physical coordinate based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of at least one object in the scene coordinate system; and sending the first physical coordinates of at least one object in the scene coordinate system to a server.

[0013] According to an embodiment of the present disclosure, based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, the second physical coordinate is corrected to obtain the first physical coordinates of at least one object in the scene coordinate system, including: based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, determining the coordinate correction value of the origin coordinates of the image coordinate system in the scene coordinate system relative to the origin coordinates of the scene coordinate system; and correcting the second physical coordinate based on the coordinate correction value to obtain the first physical coordinates of at least one object in the scene coordinate system.

[0014] According to another aspect of the present disclosure, a data processing device is provided, including: a first receiving module, used to receive the first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and the calibrated coordinates for a scene view from a terminal; the calibrated coordinates represent the size of a view area corresponding to the scene in the scene view; a division module, used to divide the scene in the scene coordinate system to obtain a plurality of grids; an allocation module, used to allocate the first physical coordinates of at least one object to the corresponding grid for each of the plurality of grids based on the first physical coordinates of at least one object in the scene coordinate system; a fusion module, used to fuse the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; a first generation module, used to generate initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene and the fused physical coordinates; and a first sending module, used to send the initial hotspot data to the terminal so that the terminal generates a thermal map within the scene based on the initial hotspot data.

[0015] According to an embodiment of the present disclosure, the first generation module includes: a first conversion unit, used to convert the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; a first determination unit, used to determine the initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and a second determination unit, used to determine the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0016] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively captured at multiple image capture moments in the scene coordinate system; the allocation module includes: a third determination unit, used to determine the number of first physical coordinates of the object contained in the images respectively captured at multiple image capture moments; and an allocation unit, used to allocate the first physical coordinates of the object contained in the image captured at each image capture moment to the corresponding grid in response to the determination that the number exceeds a preset threshold, so as to obtain the first physical coordinates of the object contained in each grid at each image capture moment.

[0017] According to an embodiment of the present disclosure, the fusion module includes: a first fusion unit, which is used to fuse the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and a second fusion unit, which is used to fuse the initial fused physical coordinates in each grid at multiple image acquisition moments to obtain fused physical coordinates.

[0018] According to another aspect of the present disclosure, a heat map generating device is provided, including: a second sending module, used to send calibrated coordinates for a scene view to a server; the calibrated coordinates represent the size of a view area corresponding to a scene in the scene view; a first acquisition module, used to obtain a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution in response to receiving initial hotspot data from the server; a first conversion module, used to convert the initial hotspot data into target hotspot data at the first page resolution based on the first height, the first width, and the original height and the original width; and a second generation module, used to generate a heat map within the scene based on the target hotspot data.

[0019] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; the first conversion module includes: a fourth determination unit, used to determine a first width ratio based on the first width and the original width; a fifth determination unit, used to determine a first height ratio based on the first height and the original height; a second conversion unit, used to convert the initial fused pixel coordinates into target pixel coordinates according to the first width ratio and the first height ratio; a third conversion unit, used to convert the initial display radius into a target display radius according to the first width ratio; and a sixth determination unit, used to determine the target hotspot data according to the target pixel coordinates and the target display radius.

[0020] According to an embodiment of the present disclosure, the heat map generating device also includes: a second acquisition module, used to acquire the second height and second width of the scene view at the second page resolution; a second receiving module, used to receive configuration information of the origin coordinates of the scene view; a calibration module, used to calibrate the view area in the scene view based on the origin coordinates of the scene view, using the second height and second width as well as the original height and original width, to obtain calibrated coordinates for the scene view.

[0021] According to an embodiment of the present disclosure, the calibration module includes: a receiving unit, used to receive a calibration operation for a view area to determine calibration position information for the view area; a seventh determination unit, used to determine a second height ratio based on the second height and the original height; an eighth determination unit, used to determine a second width ratio based on the second width and the original width; and a ninth determination unit, used to determine a calibration coordinate based on the second height ratio, the second width ratio and the calibration position information for the view area.

[0022] According to another aspect of the present disclosure, there is provided an image processing device, comprising: a third acquisition module, used to acquire an image acquired at each image acquisition moment; a determination module, used to determine the pixel coordinates of at least one object in the image in the image coordinate system; a second conversion module, used to convert the pixel coordinates of at least one object in the image coordinate system into a second physical coordinate in the scene coordinate system relative to the origin coordinates of the image coordinate system; a correction module, used to correct the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of at least one object in the scene coordinate system; and a third sending module, used to send the first physical coordinates of at least one object in the scene coordinate system to a server.

[0023] According to an embodiment of the present disclosure, the correction module includes: a tenth determination unit, which is used to determine the coordinate correction value of the origin coordinates of the image coordinate system in the scene coordinate system relative to the origin coordinates of the scene coordinate system based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system; and a correction unit, which is used to correct the second physical coordinate based on the coordinate correction value to obtain the first physical coordinates of at least one object in the scene coordinate system.

[0024] According to another aspect of the present disclosure, an electronic device is provided, including: a memory and a processor, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes the method as described above.

[0025] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method as described above.

[0026] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0027] According to another aspect of the present disclosure, a cloud server is provided, comprising the data processing device as described above.

[0028] According to another aspect of the present disclosure, a terminal is provided, comprising the heat map generating device as described above.

[0029] According to another aspect of the present disclosure, an image acquisition device is provided, comprising the image processing apparatus as described above.

[0030] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0032] Figure 1 It is a system architecture applicable to a data processing method and apparatus, a heat map generation method and apparatus, and an image processing method and apparatus according to an embodiment of the present disclosure;

[0033] Figure 2 is a flow chart of a data processing method according to an embodiment of the present disclosure;

[0034] Figure 3 is a schematic diagram of determining the physical coordinates of each grid according to an embodiment of the present disclosure;

[0035] Figure 4 is a flow chart of a method for obtaining fused physical coordinates according to an embodiment of the present disclosure;

[0036] Figure 5 is a flow chart of a method for generating a heat map according to an embodiment of the present disclosure;

[0037] Figure 6 is a schematic diagram of a process of determining calibration coordinates for a scene view according to an embodiment of the present disclosure;

[0038] Fig. 7A and Figure 7B These are the renderings of the heat maps generated at different page resolutions;

[0039] Figure 8 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0040] Fig. 9 schematically showing second physical coordinates of a plurality of objects in an image in a scene coordinate system relative to the origin coordinates of an image coordinate system;

[0041] Fig.10 is a block diagram of a data processing device according to an embodiment of the present disclosure;

[0042] Fig.11 is a block diagram of a heat map generating device according to an embodiment of the present disclosure;

[0043] Fig.12 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0044] Fig.13 It is a block diagram of an electronic device used to implement the data processing method, heat map generation method and image processing method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0045] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0046] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0047] Figure 1 It is a system architecture applicable to the data processing method and apparatus, the heat map generation method and apparatus, and the image processing method and apparatus according to the embodiments of the present disclosure. It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other environments or scenarios.

[0048] like Figure 1 As shown, the system architecture 100 may include: an image acquisition device 101 , a cloud server 102 and a terminal 103 .

[0049] In the embodiment of the present disclosure, the image acquisition device 101 can realize image acquisition and data processing functions. The image acquisition device 101 can include multiple image acquisition devices, or be connected to multiple image acquisition devices through communication.

[0050] In the embodiment of the present disclosure, the cloud server 102 can implement data processing functions.

[0051] In the embodiments of the present disclosure, the terminal 103 can be used to implement image display and data processing functions. In some embodiments, the terminal 103 can also be connected to an external display device, and the content processed by the terminal 103 can be displayed on the external display device.

[0052] In an embodiment of the present disclosure, the image acquisition device 101 may acquire an image acquired at each image acquisition moment, and based on the correspondence between the image coordinate system and the scene coordinate system, convert the pixel coordinates of at least one object in the image in the image coordinate system into the first physical coordinates in the scene coordinate system. Then, the first physical coordinates of at least one object in the scene coordinate system are sent to the cloud server 102. The terminal 103 may calibrate the view area corresponding to the scene in the scene view, and send the calibrated coordinates of the calibrated scene view to the cloud server 102. After the cloud server 102 receives the first physical coordinates of at least one object in the scene coordinate system and the calibrated coordinates for the scene view, it may generate initial hotspot data based on the first physical coordinates and the calibrated coordinates of each object, and transmit the initial hotspot data to the terminal 103. After the terminal 103 acquires the initial hotspot data returned by the cloud server 102, it may also convert the initial hotspot data to obtain target hotspot data matching the current page resolution. Then, a heat map in the scene is generated according to the target hotspot data, and the heat map is displayed.

[0053] It should be understood that Figure 1 The number of image acquisition devices 101 and cloud servers 102 in the embodiment is only for illustration. Any number of image acquisition devices 101 and cloud servers 102 may be provided according to implementation requirements.

[0054] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0055] Figure 2 is a flow chart of a data processing method according to an embodiment of the present disclosure.

[0056] like Figure 2 As shown, the data processing method 200 may include operations S210 to S260. In the embodiment of the present disclosure, the data processing method 200 may be executed by a server (eg, a cloud server) as an execution subject to perform the related operations in the embodiment of the present disclosure.

[0057] In operation S210, first physical coordinates of at least one object in an image in a scene coordinate system are received from an image acquisition device and calibrated coordinates for a scene view are received from a terminal.

[0058] In operation S220, the scene is divided in the scene coordinate system to obtain a plurality of grids.

[0059] In operation S230 , for each of the plurality of grids, first physical coordinates of the at least one object are respectively allocated to the corresponding grid based on the first physical coordinates of the at least one object in the scene coordinate system.

[0060] In operation S240, a fusion process is performed on the first physical coordinates of the objects included in the respective grids to obtain fused physical coordinates.

[0061] In operation S250 , initial hotspot data based on the image coordinate system is generated according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates.

[0062] In operation S260, the initial hotspot data is sent to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0063] According to an embodiment of the present disclosure, a server (eg, a cloud server) may receive first physical coordinates of at least one object in an image sent by an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal.

[0064] In the embodiment of the present disclosure, the first physical coordinates of at least one object in the image in the scene coordinate system are obtained by converting the pixel coordinates of at least one object in the image in the image coordinate system by capturing the image of the object in the scene at the corresponding time at a certain time interval by the image acquisition device. The time interval can be set according to actual needs and is not limited here.

[0065] In the embodiment of the present disclosure, the calibrated coordinates for the scene view can be used to characterize the size of the view area corresponding to the scene in the scene view. The area covered by the view area is the scene area where the objects concerned by each object are located.

[0066] It is understandable that the scene view may contain noise areas other than the view area. The noise area may be an area where object statistics are not required, and the area can be set according to actual conditions. In addition, the noise area may also refer to an area that is meaningless for object statistics. For example, when the terminal is used to display the view area, due to changes in page resolution or page zooming, some noise areas appear around the view area, and the area covered by the noise area does not include the scene area where the object of interest is located. In view of this, the actual size of the view area in the scene view can be calibrated by obtaining the calibration coordinates for the scene view, thereby avoiding interference from the noise area and facilitating subsequent processing.

[0067] Figure 3 Schematic diagram of determining the physical coordinates of each grid according to an embodiment of the present disclosure. Figure 3 An example process for determining the physical coordinates of each grid is described.

[0068] like Figure 3 As shown, in the scene coordinate system, the actual width X and actual height Y of the scene 1330 can be determined based on the position information of the scene 1330. Then, the scene 1330 is divided according to the preset grid size grid_length to obtain multiple grids. Among them, the grid size can be set according to actual conditions, for example, the grid size can be 0.1m, 0.2m, etc.

[0069] For example, determine the horizontal number X_totle of grids obtained by dividing the actual width X of the scene 1330, and determine the vertical number Y_totle of grids obtained by dividing the actual height Y of the scene 1330. X_totle and Y_totle are determined using the following formulas (1) and (2), respectively.

[0070] X_totle=int(math.ceil(X / grid_length) (1)

[0071] Y_totle=int(math.ceil(Y / grid_length) (2)

[0072] Wherein, math.ceil indicates that the calculation is rounded up to an integer, that is, if X / grid_length is an integer, then X_totle is equal to (X / grid_length), otherwise X_totle=X / grid_length+1.

[0073] The scene 1330 is segmented based on X_totle and Y_totle to obtain multiple grids, each of which is Figure 3 The smallest rectangle in (only for example). After obtaining multiple grids, the physical coordinates of each grid can be determined. For example, taking the physical coordinates of a grid as an example, the physical coordinates of the grid include four vertex coordinates, which are the upper left vertex coordinate W1, the upper right vertex coordinate W2, the lower right vertex coordinate W3 and the lower left vertex coordinate W4. The coordinates of each vertex are shown in formulas (3) to (6).

[0074] W1=[W1_x, W1_y]=[W_left_top_x+x_row*grid_length,

[0075] W_left_top_y+y_column*grid_length] (3)

[0076] W2=[W1_x+grid_length, W1_y] (4)

[0077] W3=[W1_x+grid_length, W1_y+grid_length] (5)

[0078] W4=[W1_x, W1_y+grid_length] (6)

[0079] In formulas (3) to (6), W_left_top_x and W_left_top_y are the horizontal and vertical coordinates of the top left vertex of the scene, respectively. The identifiers of multiple grids are a, b, c, d, e, and f from top to bottom and from left to right. Grid a corresponds to x_row=0 and y_column=0, grid b corresponds to x_row=0 and y_column=1, and grid e corresponds to x_row=1 and y_column=0.

[0080] It can be understood that calculating the physical coordinates of each grid based on the location information of the scene area, the size of the grid, and the identification of the grid can improve the calculation efficiency and accuracy of the physical coordinates of the grid.

[0081] Next, for each of the plurality of grids, based on the first physical coordinates of the at least one object in the scene coordinate system, the first physical coordinates of the at least one object are respectively allocated to the corresponding grid.

[0082] Afterwards, the first physical coordinates of the objects contained in each grid are fused to obtain fused physical coordinates.

[0083] For example, suppose the ith grid contains the first physical coordinates of m objects, i=1, 2, ..., X_totle*Y_totle, and m is a positive integer. Determine the average value of the horizontal coordinates of the m first physical coordinates in the ith grid and the average value of the ordinates of the m first physical coordinates and The following formulas (7) and (8) are used to determine respectively.

[0084]

[0085]

[0086] In formulas (7) to (8), x j and j They respectively represent the abscissa and ordinate of the j-th (j=1, 2, ..., m) first physical coordinate.

[0087] Sure and Afterwards, according to and The fused physical coordinates can be obtained

[0088] Next, based on the correspondence between the scene coordinate system and the image coordinate system, the fused physical coordinates in the scene coordinate system can be converted into initial hotspot data in the image coordinate system according to the calibrated coordinates for the scene view, the actual width and height of the scene, and the fused physical coordinates. Then, the initial hotspot data is sent to the terminal so that the terminal generates a heat map in the scene based on the initial hotspot data.

[0089] In the technical solution disclosed in the present invention, on the one hand, the actual size of the view area in the scene view is calibrated by obtaining the calibration coordinates for the scene view to avoid interference from the noise area, thereby providing a basis for improving the accuracy of the initial hotspot data. On the other hand, based on the correspondence between the above-mentioned scene coordinate system and the image coordinate system, the present invention realizes the conversion of physical coordinates into pixel coordinates, thereby ensuring the accuracy of the initial hotspot data, and further ensuring the accuracy of the heat map.

[0090] As described above, the first physical coordinates are obtained by converting the pixel coordinates of at least one object in the image coordinate system based on the image acquisition device capturing the image of the object in the scene at the corresponding time at a certain time interval. Afterwards, the server performs fusion processing based on the first physical coordinates of at least one object to obtain fused physical coordinates.

[0091] In some embodiments, the first physical coordinates of at least one object in the image in the scene coordinate system may include the first physical coordinates of the object contained in the images respectively acquired at multiple image acquisition times in the scene coordinate system. If the number of first physical coordinates received by the server is too large, it may exceed the storage capacity and computing capacity of the server, resulting in data overflow, thereby affecting the normal generation of initial hotspot data.

[0092] In the disclosed embodiment, the first physical coordinates obtained at multiple image acquisition moments can be processed in segments according to time periods, so that the server only needs to fuse the first physical coordinates corresponding to each time period each time to obtain the initial fused physical coordinates. After that, the initial physical coordinates after aggregation of each segment are fused to obtain the fused physical coordinates. In this way, the number of first physical coordinates processed by the server each time can be reduced, thereby solving the problem of memory overflow and failure to generate initial hotspot data normally on the server.

[0093] Reference below Figure 4 The process of segmentation is explained with examples.

[0094] Figure 4 is a flowchart of a method for obtaining fused physical coordinates according to an embodiment of the present disclosure.

[0095] like Figure 4 As shown, the method for acquiring fused physical coordinates may include operations S331 to S334 and operations S341 to S343.

[0096] In operation S331 , the number of first physical coordinates of objects contained in images respectively acquired at a plurality of image acquisition moments is determined.

[0097] In operation S332, it is determined whether the number of first physical coordinates of objects contained in the images collected at the multiple image collection moments exceeds a preset threshold. If it is determined that the number exceeds the preset threshold, operations S333 and S341-S342 are performed; otherwise, operations S334 and S343 are performed.

[0098] In operation S333, the first physical coordinates of the objects contained in the images collected at each image collection moment are allocated to corresponding grids to obtain the first physical coordinates of the objects contained in each grid at each image collection moment.

[0099] In operation S334, first physical coordinates of objects contained in images acquired at multiple image acquisition moments are allocated to corresponding grids to obtain first physical coordinates of objects contained in each grid.

[0100] In operation S341, the first physical coordinates of the objects contained in each grid at each image acquisition moment are fused to obtain the initial fused physical coordinates of each grid at each image acquisition moment.

[0101] In operation S342, the initial fused physical coordinates in each network at the time of multiple image acquisition are fused to obtain fused physical coordinates.

[0102] In operation S343, the first physical coordinates of the objects included in the respective grids are fused to obtain fused physical coordinates.

[0103] According to an embodiment of the present disclosure, if it is determined that the number of first physical coordinates of objects contained in images respectively collected at multiple image acquisition moments exceeds a preset threshold, it means that the number of first physical coordinates received by the server exceeds the processing capacity of the server. At this time, the first physical coordinates obtained at multiple image acquisition moments can be processed in segments according to time periods to reduce the number of first physical coordinates processed by the server each time. If it is determined that the number of first physical coordinates of objects contained in images respectively collected at multiple image acquisition moments does not exceed the preset threshold, it means that the storage capacity and computing power of the server can meet the processing requirements for the first physical coordinates. At this time, the first physical coordinates corresponding to the multiple image acquisition moments can be directly fused to obtain fused physical coordinates.

[0104] It should be noted that, in the disclosed embodiments, the preset threshold value can be set according to actual conditions and is not specifically limited.

[0105] In addition, when the first physical coordinates obtained at multiple image acquisition moments are processed in sections, the first physical coordinates may also be processed in sections according to preset time periods based on actual conditions, which is not limited here.

[0106] According to an embodiment of the present disclosure, the above-mentioned initial hotspot data may be generated in the following manner.

[0107] For example, the fused physical coordinates may be converted to initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene.

[0108] For the sake of simplicity, the following description is made by taking the view area in the scene view as a regular rectangular area as an example. It should be understood that the solution of the present disclosure is not limited to this.

[0109] Assume that the calibration coordinates for the scene view include two vertex coordinates, which are the upper left vertex coordinates and the lower right vertex coordinates corresponding to the view area. Taking the upper left vertex of the scene view as the origin coordinate, the upper left vertex coordinates corresponding to the view area can be expressed as (left_top_x, left_top_y), and the lower right vertex coordinates can be expressed as (right_bottom_x, right_bottom_y).

[0110] For example, the following formulas (9) to (10) may be used to determine the initial fused pixel coordinates.

[0111]

[0112]

[0113] In formulas (9) to (10), pixel_x and pixel_y are the horizontal and vertical coordinates of the initial fused pixel coordinates, respectively. are the horizontal and vertical coordinates of the fused physical coordinates, and X and Y are the actual width and height of the scene respectively.

[0114] In addition, the initial display radius of the fused pixel coordinates may be determined according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene.

[0115] For example, the initial display radius may be determined using the following formula (11).

[0116] pixel_radius=Math.max((right_bottom_x-left_top_x) / X, (right_bottom_y-left_top_y) / Y) (11)

[0117] In formula (11), pixel_radius represents the initial display radius.

[0118] Afterwards, the initial fused pixel coordinates and the initial display radius are determined as initial hotspot data.

[0119] In the embodiments of the present disclosure, for example, the corresponding relationship described in the above formulas (9) to (11) can be used to convert the physical coordinates into pixel coordinates based on the calibrated coordinates of the scene view, the actual width and actual height of the scene, and the fused physical coordinates, thereby ensuring the accuracy of the initial hotspot data and further ensuring the accuracy of the heat map.

[0120] Figure 5 is a flow chart of a method for generating a heat map according to an embodiment of the present disclosure.

[0121] like Figure 5 As shown, the heat map generation method 400 may include operations S410 to S440. In the embodiment of the present disclosure, the heat map generation method 400 may be performed by a terminal as an execution subject to perform related operations in the embodiment of the present disclosure.

[0122] In operation S410, the calibrated coordinates for the scene view are sent to the server.

[0123] In operation S420, in response to receiving the initial hotspot data from the server, a first height and a first width of the scene view at the first page resolution and an original height and an original width of the scene view at the original page resolution are obtained.

[0124] In operation S430, the initial hotspot data is converted into target hotspot data at a first page resolution based on the first height, the first width, and the original height and the original width.

[0125] In operation S440 , a heat map within the scene is generated based on the target hotspot data.

[0126] According to an embodiment of the present disclosure, the terminal can calibrate the actual size of the view area corresponding to the scene in the scene view at the second page resolution to obtain the calibrated coordinates for the scene view. The second page resolution is used to indicate the page resolution corresponding to the scene view when the scene view is calibrated.

[0127] After determining the calibrated coordinates for the scene view, the terminal can send the calibrated coordinates to a server (eg, a cloud server) so that the server can convert physical coordinates to pixel coordinates based on the calibrated coordinates and generate initial hotspot data based on the image coordinate system.

[0128] It is understandable that when the page resolution changes during the process of generating a heat map on the terminal, the display effect of the hotspot data on the heat map will be affected. For example, the change of the page resolution will cause the hotspot data to shift, resulting in the hotspot data being incompatible with the current page resolution.

[0129] In order to solve the offset of hotspot data caused by different page resolutions, in the embodiment of the present disclosure, after receiving the initial hotspot data from the server, the terminal can obtain the first height and first width of the scene view at the first page resolution and the original height and original width of the scene view at the original page resolution. Among them, the first page resolution is used to indicate the page resolution corresponding to the scene view when the heat map is to be generated, that is, the current page resolution. The original page resolution is used to indicate the page resolution corresponding to the scene view before the scene view is calibrated.

[0130] Afterwards, the terminal can convert the initial hotspot data into target hotspot data at the first page resolution based on the first height, the first width, and the original height and the original width. Then, based on the target hotspot data, a heat map within the scene is generated. In the above manner, the initial hotspot data can be converted into target hotspot data that matches the current page resolution, which is conducive to displaying a page adapted to the current page resolution, thereby ensuring the accuracy of the heat map.

[0131] According to an embodiment of the present disclosure, the calibration coordinates for the scene view may be determined in the following manner.

[0132] It should be noted that, for the sake of simplicity, the following takes the view area in the scene view as a regular rectangular area as an example, and refers to Figure 6 The process of determining the calibration coordinates for the scene view is described by way of example. It should be understood that Figure 6 The contents shown in the figure are merely exemplary and are intended to help understand the solutions of the present disclosure, but the present disclosure is not limited thereto.

[0133] Figure 6 It is a schematic diagram of a process of determining calibration coordinates for a scene view according to an embodiment of the present disclosure.

[0134] After the terminal obtains the scene view, the display ratio of the scene view can be scaled and adjusted on the display interface of the terminal to obtain a good display effect. Figure 6As shown, the scene view 510 after zoom adjustment includes a view area 520 corresponding to the scene and a noise area (such as the area between the scene view 510 and the view area 520) other than the view area 520. The area covered by the noise area does not include the scene area where the object of interest is located.

[0135] In order to avoid interference from the noise area, in the embodiment of the present disclosure, the actual size of the viewing area 520 may be calibrated.

[0136] For example, the second height scaling_height and the second width scaling_width of the scene view 510 at the second page resolution may be obtained, wherein the second page resolution refers to the page resolution corresponding to the scene view after the display ratio of the scene view is scaled and adjusted.

[0137] Afterwards, configuration information of the origin coordinates of the scene view 510 is received. For example, according to the configuration information, it is determined that the origin coordinates of the scene view 510 are located at the upper left vertex of the scene view 510, that is, point O.

[0138] Next, based on the origin coordinates of the scene view 510 , the view area 520 in the scene view 510 may be calibrated using the second height and the second width as well as the original height and the original width to obtain calibrated coordinates for the scene view 510 .

[0139] For example, a calibration operation for view area 520 is received to determine calibration position information for view area 520. For example, according to the calibration operation for view area 520, multiple calibration points in view area 520 may be determined, such as calibration point 521, calibration point 522, calibration point 523, and calibration point 524. The multiple calibration points may be used to calibrate the actual size of view area 520.

[0140] Each calibration point corresponds to a calibration position information, and the calibration position information includes the width click_x and height click_y corresponding to the calibration point, wherein the width click_x and the height click_y refer to the width and height relative to the position of point O, respectively.

[0141] Next, a second height ratio may be determined according to the second height and the original height, a second width ratio may be determined according to the second width and the original width, and then a calibration coordinate may be determined based on the second height ratio, the second width ratio, and the calibration position information for the view area.

[0142] For example, the following formulas (12) to (13) may be used to determine the calibration coordinates of each calibration point.

[0143] result_x=click_x*(resolution_width / scaling_width) (12)

[0144] result_y=click_y*(resolution_height / scaling_height) (13)

[0145] In the above formulas (12) to (13), resolution_width and resolution_height respectively represent the original width and original height of the scene view at the original page resolution, scaling_width and scaling_height respectively represent the second width and second height of the scene view at the second page resolution, click_x and click_y represent the calibration position information of each calibration point, and result_x and result_y respectively represent the horizontal and vertical coordinates of the calibration coordinates of each calibration point.

[0146] Based on the above formulas (12) to (13), the calibration coordinates corresponding to calibration points 521 to 524 can be determined. Based on the calibration coordinates corresponding to calibration points 521 to 524, the size of view area 520 can be determined. Thus, the size of view area 520 is calibrated.

[0147] In some embodiments, the size of the view area 520 may be calibrated according to the calibration coordinates corresponding to the calibration points 521 and 523 or the calibration coordinates corresponding to the calibration points 522 and 524, respectively. The specific size may be selected according to actual needs.

[0148] According to an embodiment of the present disclosure, the initial hotspot data includes initial fusion pixel coordinates and an initial display radius. Based on the first height, the first width, and the original height and the original width, converting the initial hotspot data into target hotspot data at the first page resolution may include the following operations.

[0149] For example, based on the first width and the original width, a first width ratio is determined. Based on the first height and the original height, a first height ratio is determined. Then, according to the first width ratio and the first height ratio, the initial fused pixel coordinates are converted to the target pixel coordinates. According to the first width ratio, the initial display radius is converted to the target display radius. Then, according to the target pixel coordinates and the target display radius, the target hotspot data is determined.

[0150] For example, the following formulas (14) to (16) may be used to determine the target pixel coordinates and the target display radius.

[0151] current_pixel_x=pixel_x*current_resolution_width / resolution_width (14)

[0152] current_pixel_y=pixel_y*current_resolution_height / resolution_height (15)

[0153] current_pixel_radius=pixel_radius*current_resolution_width / resolution_width (16)

[0154] In formulas (14) to (16), current_pixel_x and current_pixel_y represent the horizontal and vertical coordinates of the target pixel coordinates, respectively, current_pixel_radius represents the target display radius, pixel_x and pixel_y represent the horizontal and vertical coordinates of the initial fused pixel coordinates, respectively, pixel_radius represents the initial display radius, resolution_width and resolution_height represent the original width and original height of the scene view at the original page resolution, respectively, and current_resolution_width and current_resolution_height represent the first width and first height of the scene view at the first page resolution, respectively.

[0155] Based on the above formulas (14) to (16), the initial hotspot data can be converted into target hotspot data that matches the current page resolution, thereby achieving page display adapted to different page resolutions.

[0156] Fig. 7A and Figure 7B These are the effects of heat maps generated at different page resolutions.

[0157] Fig. 7A This is a heat map generated at a page resolution of 854*480. Figure 7B This is a heat map generated at a page resolution of 1916*1078. Fig. 7A and Figure 7BAs shown, the heat map generated by the solution of the present disclosure can be adapted to pages with different page resolutions for display. In addition, based on the heat map, the degree of attention paid to each area in the scene by the object can be clearly and intuitively displayed. In the figure, the light spots represent objects. If the number of light spots in a certain area is greater, the color of the corresponding light spots is darker, indicating that the area is more concerned.

[0158] Figure 8 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0159] like Figure 8 As shown, the image processing method 700 may include operations S710 to S750. In the embodiment of the present disclosure, the image processing method 700 may be performed by an image acquisition device as an execution subject to perform the related operations in the embodiment of the present disclosure.

[0160] In operation S710, an image captured at each image capturing moment is acquired.

[0161] In operation S720, pixel coordinates of at least one object in the image in the image coordinate system are determined.

[0162] In operation S730, pixel coordinates of at least one object in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to origin coordinates of the image coordinate system.

[0163] In operation S740, based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, the second physical coordinates are corrected to obtain first physical coordinates of at least one object in the scene coordinate system.

[0164] In operation S750, first physical coordinates of at least one object in the scene coordinate system are sent to the server.

[0165] According to an embodiment of the present disclosure, the image acquisition device may include multiple image acquisition devices. The multiple image acquisition devices (such as cameras) are arranged at corresponding positions in the scene. The multiple image acquisition devices may cover the area where the scene is located.

[0166] The image acquisition device can acquire images acquired by multiple image acquisition devices at each image acquisition time. The image acquisition time can be set according to actual conditions and is not limited here. After acquiring the image acquired at each image acquisition time, the image acquisition device can determine the pixel coordinates of at least one object in the image in the image coordinate system.

[0167] Afterwards, based on the correspondence between the image coordinate system and the scene coordinate system, the pixel coordinates of the at least one object in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0168] Next, based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, the second physical coordinates are corrected to obtain the first physical coordinates of at least one object in the scene coordinate system. It should be noted that the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system can be the same point or different points, depending on the actual scene setting.

[0169] Then, the first physical coordinates of at least one object in the scene coordinate system are sent to a server (eg, a cloud server) so that the server generates initial hotspot data based on the first physical coordinates.

[0170] According to an embodiment of the present disclosure, the pixel coordinates of at least one object in the image are converted and corrected by utilizing an image acquisition device to obtain the first physical coordinates of each object in the scene coordinate system, and the first physical coordinates of each object in the scene coordinate system are sent to the server. This is conducive to real-time processing of the image to obtain corresponding hotspot data, thereby facilitating rapid acquisition of the density distribution of objects in the scene.

[0171] According to an embodiment of the present disclosure, the corresponding relationship between the image coordinate system and the scene coordinate system may be determined in the following manner.

[0172] For example, four marking points may be determined in the image, and the pixel coordinates of each of the four marking points in the image coordinate system and the third physical coordinates of the four marking points relative to the origin coordinates of the image coordinate system may be obtained.

[0173] The pixel coordinates in the image coordinate system are converted into physical coordinates in the scene coordinate system (world coordinate system) using the following formula (17).

[0174]

[0175] In formula (17), (x w ,y w , z w ) represents the physical coordinates in the scene coordinate system, (x im ,y im ) represents the physical coordinate (x w ,y w , z w ) The pixel coordinates in the image coordinate system obtained by conversion, represents the affine matrix, a 11 ~a 32 represents the parameters in the affine matrix. In which, let the image plane be 1, and the coordinates have only one scale factor, so z im =1, a 33 =1.

[0176] According to formula (17), the following formulas (18) to (20) can be obtained.

[0177] x w =a 11 *x im +a 12 *y im +a 13 (18)

[0178] y w =a 21 *x im +a 22 *y im +a 23 (19)

[0179] z w =a 31 *x im +a 32 *y im +1 (20)

[0180] If you want to get z w The physical coordinates in the plane (x world ,y world ), the following formula (21) can be obtained by the projected triangle similarity method.

[0181]

[0182] According to formulas (18) to (21), the physical coordinates (x world ,y world ). world and world It can be expressed as follows.

[0183]

[0184]

[0185] According to the pixel coordinates of the above four marking points in the image coordinate system, the third physical coordinates of the four marking points relative to the origin coordinates of the image coordinate system, and formulas (22) to (23), the parameter a in the affine matrix can be determined: 11 ~a 32 .

[0186] The above parameter a can be determined using the following formula (24): 11 ~a 32 .

[0187]

[0188] Determine the above parameter a 11 ~a 32 Afterwards, according to formulas (22) to (23), the correspondence between the image coordinate system and the scene coordinate system can be determined. Based on the correspondence, i.e., formulas (22) to (23), the pixel coordinates of at least one object in the image coordinate system can be converted into a second physical coordinate in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0189] Fig. 9 The second physical coordinates of multiple objects in the image in the scene coordinate system relative to the origin coordinates of the image coordinate system are schematically shown. Fig. 9 The process of correcting the second physical coordinate is described by way of example.

[0190] Fig. 9 Two coordinate systems are shown. Among them, the coordinate system with the origin coordinates of point O1 is the scene coordinate system (xO1y), and the coordinate system with the origin coordinates of point O2 is the image coordinate system (uO2v). The origin coordinates O1 and the origin coordinates O2 can be the same point (i.e., O1 and O2 coincide), or different points (i.e., O1 and O2 do not coincide), which is not limited here. For the sake of convenience, the following description is taken as an example that the origin coordinates O1 and the origin coordinates O2 are different points.

[0191] Assume that the image collected at each image collection moment includes 4 objects, such as object A, object B, object C and object D. Then, determine the pixel coordinates of each of the 4 objects in the image coordinate system, for example, the pixel coordinates corresponding to objects A to D are A(x1im, y1im), B(x2im, y2im), C(x3im, y3im), and D(x4im, y4im).

[0192] Next, based on the above formulas (22) to (23), the pixel coordinates of the four objects in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system. Fig. 9 As shown, it can be obtained that the second physical coordinates of object A relative to the origin coordinate O2 of the image coordinate system are (x1, y1), the second physical coordinates of object B relative to the origin coordinate O2 of the image coordinate system are (x2, y2), the second physical coordinates of object C relative to the origin coordinate O2 of the image coordinate system are (x3, y3), and the second physical coordinates of object D relative to the origin coordinate O2 of the image coordinate system are (x4, y4).

[0193] Since there is a coordinate error between the origin coordinate O2 of the image coordinate system and the origin coordinate O1 of the scene coordinate system, it is necessary to correct the second physical coordinates of each object so as to convert the second physical coordinates into first physical coordinates relative to the origin coordinate O1 of the scene coordinate system.

[0194] For example, based on the correspondence between the origin coordinate O2 of the image coordinate system and the origin coordinate O1 of the scene coordinate system, the coordinate correction value of the origin coordinate O2 of the image coordinate system relative to the origin coordinate O1 of the scene coordinate system in the scene coordinate system can be determined. For example, the coordinate correction value can be determined according to the difference in the horizontal coordinate and the difference in the vertical coordinate between the origin coordinate O2 and the origin coordinate O1.

[0195] Afterwards, the second physical coordinates are corrected based on the coordinate correction value to obtain the first physical coordinates of the at least one object in the scene coordinate system.

[0196] For example, assuming that the coordinate correction value is (Δx, Δy), taking object A as an example, the second physical coordinate (x1, y1) of object A can be corrected using the following formula (25) to obtain the first physical coordinate (x1', y1') of object A in the scene coordinate system.

[0197] x1'=x1+Δx, y1'=y1+Δy (25)

[0198] Similarly, based on the above method, the first physical coordinates of each of objects B to D in the scene coordinate system can be obtained.

[0199] According to an embodiment of the present disclosure, the second physical coordinates are corrected based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates, thereby improving the accuracy of the first physical coordinates.

[0200] Fig.10 is a block diagram of a data processing device according to an embodiment of the present disclosure.

[0201] like Fig.10 As shown, the data processing device 900 includes: a first receiving module 910, a dividing module 920, an allocating module 930, a merging module 940, a first generating module 950 and a first sending module 960.

[0202] The first receiving module 910 is used to receive the first physical coordinates of at least one object in the image from the image acquisition device in the scene coordinate system and the calibrated coordinates for the scene view from the terminal; the calibrated coordinates represent the size of the view area corresponding to the scene in the scene view.

[0203] The division module 920 is used to divide the scene in the scene coordinate system to obtain multiple grids.

[0204] The allocation module 930 is used to allocate the first physical coordinates of at least one object to the corresponding grid for each of the multiple grids based on the first physical coordinates of the at least one object in the scene coordinate system.

[0205] The fusion module 940 is used to perform fusion processing on the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates.

[0206] The first generating module 950 is used to generate initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene, and the fused physical coordinates.

[0207] The first sending module 960 is used to send the initial hotspot data to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0208] According to an embodiment of the present disclosure, the first generation module 950 includes: a first conversion unit, a first determination unit, and a second determination unit. The first conversion unit is used to convert the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; the first determination unit is used to determine the initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and the second determination unit is used to determine the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0209] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively collected at multiple image collection moments in the scene coordinate system; the allocation module 930 includes: a third determination unit and an allocation unit. The third determination unit is used to determine the number of first physical coordinates of the object contained in the images respectively collected at multiple image collection moments; and the allocation unit is used to allocate the first physical coordinates of the object contained in the image collected at each image collection moment to the corresponding grid in response to determining that the number exceeds a preset threshold, so as to obtain the first physical coordinates of the object contained in each grid at each image collection moment.

[0210] According to an embodiment of the present disclosure, the fusion module 940 includes: a first fusion unit and a second fusion unit. The first fusion unit is used to fuse the first physical coordinates of the object contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and the second fusion unit is used to fuse the initial fused physical coordinates in each grid at multiple image acquisition moments to obtain the fused physical coordinates.

[0211] Fig.11 is a block diagram of a heat map generating device according to an embodiment of the present disclosure.

[0212] like Fig.11 As shown, the heat map generating device 1000 includes: a second sending module 1010 , a first acquiring module 1020 , a first converting module 1030 and a second generating module 1040 .

[0213] The second sending module 1010 is used to send the calibrated coordinates for the scene view to the server; the calibrated coordinates represent the size of the view area corresponding to the scene in the scene view.

[0214] The first acquisition module 1020 is used to acquire a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution in response to receiving initial hotspot data from the server.

[0215] The first conversion module 1030 is used to convert the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, and the original height and the original width.

[0216] The second generation module 1040 is used to generate a heat map within the scene based on the target hotspot data.

[0217] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; the first conversion module 1030 includes: a fourth determination unit, a fifth determination unit, a second conversion unit, a third conversion unit, and a sixth determination unit. The fourth determination unit is used to determine a first width ratio based on the first width and the original width; the fifth determination unit is used to determine a first height ratio based on the first height and the original height; the second conversion unit is used to convert the initial fused pixel coordinates into target pixel coordinates according to the first width ratio and the first height ratio; the third conversion unit is used to convert the initial display radius into a target display radius according to the first width ratio; and the sixth determination unit is used to determine the target hotspot data according to the target pixel coordinates and the target display radius.

[0218] According to an embodiment of the present disclosure, the heat map generation device 1000 further includes: a second acquisition module, a second receiving module and a calibration module. The second acquisition module is used to acquire the second height and the second width of the scene view at the second page resolution; the second receiving module is used to receive configuration information for the origin coordinates of the scene view; the calibration module is used to calibrate the view area in the scene view based on the origin coordinates of the scene view, using the second height and the second width and the original height and the original width, to obtain the calibrated coordinates for the scene view.

[0219] According to an embodiment of the present disclosure, the calibration module includes: a receiving unit, a seventh determining unit, an eighth determining unit, and a ninth determining unit. The receiving unit is used to receive a calibration operation for a view area to determine calibration position information for the view area; the seventh determining unit is used to determine a second height ratio according to the second height and the original height; the eighth determining unit is used to determine a second width ratio according to the second width and the original width; and the ninth determining unit is used to determine calibration coordinates based on the second height ratio, the second width ratio, and the calibration position information for the view area.

[0220] Fig.12 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0221] like Fig.12 As shown, the image processing device 1100 includes: a third acquisition module 1110 , a determination module 1120 , a second conversion module 1130 , a correction module 1140 and a third sending module 1150 .

[0222] The third acquisition module 1110 is used to acquire the image acquired at each image acquisition moment.

[0223] The determination module 1120 is used to determine the pixel coordinates of at least one object in the image in the image coordinate system.

[0224] The second conversion module 1130 is used to convert the pixel coordinates of at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0225] The correction module 1140 is used to correct the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of at least one object in the scene coordinate system.

[0226] The third sending module 1150 is used to send the first physical coordinates of at least one object in the scene coordinate system to the server.

[0227] According to an embodiment of the present disclosure, the correction module 1140 includes: a tenth determining unit and a correction unit. The tenth determining unit is used to determine the coordinate correction value of the origin coordinate of the image coordinate system in the scene coordinate system relative to the origin coordinate of the scene coordinate system based on the correspondence between the origin coordinate of the image coordinate system and the origin coordinate of the scene coordinate system; and the correction unit is used to correct the second physical coordinate based on the coordinate correction value to obtain the first physical coordinate of at least one object in the scene coordinate system.

[0228] It should be noted that the implementation methods, technical problems solved, functions realized, and technical effects achieved of each module / unit / sub-unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0229] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (for example, including but not limited to user personal information) shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0230] In the technical solution of the present disclosure, before obtaining or collecting relevant data, the authorization or consent of the data owner is obtained.

[0231] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0232] Fig.13 A block diagram of an electronic device suitable for implementing a data processing method, a heat map generating method, and an image processing method according to an embodiment of the present disclosure is schematically shown.

[0233] like Fig.13 As shown, the electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include an onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0234] In RAM 1203, various programs and data required for the operation of electronic device 1200 are stored. Processor 1201, ROM 1202 and RAM 1203 are connected to each other via bus 1204. Processor 1201 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1202 and / or RAM 1203. It should be noted that the program can also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0235] According to an embodiment of the present disclosure, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may further include one or more of the following components connected to the I / O interface 1205: an input portion 1206 including a keyboard, a mouse, etc.; an output portion 1206 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1208 including a hard disk, etc.; and a communication portion 1209 including a network interface card such as a LAN card, a modem, etc. The communication portion 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage portion 1208 as needed.

[0236] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0237] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1202 and / or RAM 1203 described above and / or one or more memories other than ROM 1202 and RAM 1203.

[0238] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data processing method, heat map generation method and image processing method provided in the embodiments of the present disclosure.

[0239] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1201. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0240] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1209, and / or installed from the removable medium 1211. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0241] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0242] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0243] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0244] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0245] The embodiments of the present disclosure are described above. However, these embodiments are only for the purpose of illustration and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: Receiving first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; Dividing the scene in a scene coordinate system to obtain a plurality of grids; For each of the plurality of grids, based on the first physical coordinate of the at least one object in the scene coordinate system, respectively assigning the first physical coordinate of the at least one object to the corresponding grid; Performing fusion processing on the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; Generate initial hotspot data based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates; as well as The initial hotspot data is sent to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

2. The method according to claim 1, wherein: The generating of initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates includes: According to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, the fused physical coordinates are converted into initial fused pixel coordinates based on an image coordinate system; Determining an initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and The initial fused pixel coordinates and the initial display radius are determined as the initial hotspot data.

3. The method according to claim 1 or 2, wherein: The first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively acquired at a plurality of image acquisition moments in the scene coordinate system; The allocating the first physical coordinates of the at least one object to corresponding grids based on the first physical coordinates of the at least one object in the scene coordinate system comprises: Determine the number of first physical coordinates of objects contained in the images respectively acquired at the plurality of image acquisition moments; as well as In response to determining that the number exceeds a preset threshold, the first physical coordinates of the objects contained in the image captured at each image capture moment are allocated to corresponding grids to obtain the first physical coordinates of the objects contained in each grid at each image capture moment.

4. The method according to claim 3, wherein: The fusing process of the first physical coordinates of the objects contained in each grid to obtain the fused physical coordinates includes: Fusing the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates of each grid at each image acquisition moment; and The initial fused physical coordinates in each network at the time of multiple image acquisition are fused to obtain the fused physical coordinates.

5. A method for generating a heat map, comprising: Send the calibration coordinates for the scene view to the server; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; In response to receiving the initial hotspot data from the server, obtaining a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; Based on the first height, the first width, the original height, and the original width, converting the initial hotspot data into target hotspot data at a first page resolution; as well as Based on the target hotspot data, a heat map within the scene is generated.

6. The method according to claim 5, wherein: The initial hotspot data includes initial fusion pixel coordinates and an initial display radius; and converting the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, the original height, and the original width includes: Determining a first width ratio based on the first width and the original width; Determining a first height ratio based on the first height and the original height; Converting the initial fused pixel coordinates to the target pixel coordinates according to the first width ratio and the first height ratio; Converting the initial display radius into a target display radius according to the first width ratio; and The target hotspot data is determined according to the target pixel coordinates and the target display radius.

7. The method according to claim 5, further comprising: Get the second height and the second width of the scene view at the second page resolution; Receiving configuration information of origin coordinates for the scene view; Based on the origin coordinates of the scene view, the view area in the scene view is calibrated using the second height and the second width as well as the original height and the original width to obtain calibrated coordinates for the scene view.

8. The method according to claim 7, wherein: The step of calibrating the view area in the scene view by using the second height and the second width as well as the original height and the original width to obtain the calibrated coordinates for the scene view includes: receiving a calibration operation for the viewing area to determine calibration position information for the viewing area; Determining a second height ratio according to the second height and the original height; determining a second width ratio according to the second width and the original width; and The calibration coordinates are determined based on the second height ratio, the second width ratio, and calibration position information for the view area.

9. An image processing method, comprising: Obtain the image collected at each image collection moment; determining pixel coordinates of at least one object in the image in an image coordinate system; Converting pixel coordinates of the at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system; Based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, correcting the second physical coordinates to obtain the first physical coordinates of the at least one object in the scene coordinate system; and The first physical coordinates of the at least one object in the scene coordinate system are sent to the server.

10. The method according to claim 9, wherein: The correcting the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of the at least one object in the scene coordinate system comprises: Determining, based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, a coordinate correction value of the origin coordinates of the image coordinate system relative to the origin coordinates of the scene coordinate system in the scene coordinate system; and The second physical coordinates are corrected based on the coordinate correction value to obtain the first physical coordinates of the at least one object in the scene coordinate system.

11. A data processing device, comprising: A first receiving module, configured to receive first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal; the calibrated coordinates represent a size of a view area corresponding to a scene in the scene view; A division module, used for dividing the scene in a scene coordinate system to obtain a plurality of grids; an allocation module, configured to allocate the first physical coordinate of the at least one object to a corresponding grid respectively based on the first physical coordinate of the at least one object in the scene coordinate system for each grid in the plurality of grids; A fusion module, used for fusing the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; A first generating module, configured to generate initial hotspot data based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates; as well as The first sending module is used to send the initial hotspot data to the terminal so that the terminal generates a heat map in the scene based on the initial hotspot data.

12. The device according to claim 11, wherein The first generation module comprises: A first conversion unit, configured to convert the fused physical coordinates into initial fused pixel coordinates based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; A first determining unit, configured to determine an initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and The second determining unit is used to determine the initial fusion pixel coordinates and the initial display radius as the initial hotspot data.

13. The device according to claim 11 or 12, wherein: The first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively acquired at a plurality of image acquisition moments in the scene coordinate system; The allocation module comprises: A third determining unit, configured to determine the number of first physical coordinates of objects contained in the images respectively acquired at the plurality of image acquisition moments; as well as An allocating unit is used to allocate the first physical coordinates of the objects contained in the images collected at each image collection moment to the corresponding grids in response to determining that the number exceeds a preset threshold, so as to obtain the first physical coordinates of the objects contained in each grid at each image collection moment.

14. The device according to claim 13, wherein: The fusion module includes: A first fusion unit is used to fuse the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and The second fusion unit is used to fuse the initial fused physical coordinates in each network at the time of multiple image acquisition to obtain the fused physical coordinates.

15. A heat map generating device, comprising: A second sending module, used for sending the calibration coordinates for the scene view to the server; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; A first acquisition module, configured to acquire, in response to receiving the initial hotspot data from the server, a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; A first conversion module, configured to convert the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, the original height, and the original width; as well as The second generating module is used to generate a heat map within the scene based on the target hotspot data.

16. The device according to claim 15, wherein: The initial hotspot data includes initial fusion pixel coordinates and initial display radius; The first conversion module comprises: a fourth determining unit, configured to determine a first width ratio based on the first width and the original width; a fifth determining unit, configured to determine a first height ratio based on the first height and the original height; A second conversion unit, configured to convert the initial fused pixel coordinates into the target pixel coordinates according to the first width ratio and the first height ratio; A third conversion unit, configured to convert the initial display radius into a target display radius according to the first width ratio; and The sixth determination unit is used to determine the target hotspot data according to the target pixel coordinates and the target display radius.

17. The apparatus according to claim 15, further comprising: A second acquisition module, used to acquire a second height and a second width of the scene view at a second page resolution; A second receiving module, used to receive configuration information of the origin coordinates of the scene view; A calibration module is used to calibrate the view area in the scene view based on the origin coordinates of the scene view by using the second height and the second width as well as the original height and the original width to obtain calibration coordinates for the scene view.

18. The device according to claim 17, wherein: The calibration module comprises: A receiving unit, configured to receive a calibration operation for the viewing area to determine calibration position information for the viewing area; a seventh determining unit, configured to determine a second height ratio according to the second height and the original height; an eighth determining unit, configured to determine a second width ratio according to the second width and the original width; and A ninth determining unit is configured to determine the calibration coordinates based on the second height ratio, the second width ratio, and the calibration position information for the viewing area.

19. An image processing device, comprising: The third acquisition module is used to acquire the image collected at each image collection moment; A determination module, configured to determine pixel coordinates of at least one object in the image in an image coordinate system; A second conversion module, configured to convert pixel coordinates of the at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system; a correction module, configured to correct the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, so as to obtain the first physical coordinates of the at least one object in the scene coordinate system; as well as The third sending module is used to send the first physical coordinates of the at least one object in the scene coordinate system to the server.

20. The device according to claim 19, wherein The correction module comprises: a tenth determining unit, configured to determine, based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, a coordinate correction value of the origin coordinates of the image coordinate system relative to the origin coordinates of the scene coordinate system in the scene coordinate system; and A correction unit is used to correct the second physical coordinate based on the coordinate correction value to obtain the first physical coordinate of the at least one object in the scene coordinate system.

21. An electronic device comprising a memory and a processor, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

24. A cloud server, comprising: The data processing device according to any one of claims 11 to 14.

25. A terminal, comprising: The thermal map generating device according to any one of claims 15 to 18.

26. An image acquisition device, comprising: The image processing device according to any one of claims 19 to 20.