Data processing method, heat map generation method, and device

By receiving and fusing the physical coordinates and calibration coordinates sent by the image acquisition device and the terminal, and generating and adapting the thermal map, the problems of poor heat map display and low data processing efficiency in the prior art are solved, and the resolution on different pages is achieved Accurate heat map display and efficient data processing at rate.

WO2024244734A1PCT designated stage expired Publication Date: 2024-12-05BOE TECHNOLOGY GROUP CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/088016
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-04-16
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The prior art is difficult to effectively generate and display heat maps of the degree of object attention in the scene, especially the heat map display effect is poor at different page resolutions, and the data processing efficiency is low, which can easily lead to hot spot data offset and computing power overload.

Method used

By receiving physical coordinates and calibration coordinates sent by image acquisition equipment and terminals, dividing scene coordinate systems, fusing object coordinates, generating initial hotspot data, and performing page resolution adaptation on the terminal and server side, converting hotspot data to generate accurate thermal forces picture.

Benefits of technology

It realizes accurate display of heat maps at different page resolutions, improves data processing efficiency, avoids hot spot data offset and computing power overload, and ensures the accuracy and stability of heat maps.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024088016_05122024_PF_FP_ABST
    Figure CN2024088016_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present disclosure are a data processing method, a heat map generation method, and a device. The data processing method comprises: receiving a first physical coordinate of at least one object in an image from an image acquisition device in a scene coordinate system as well as a calibration coordinate for a scene view from the terminal, the calibration coordinate representing the size of a view area corresponding to the scene in the scene view; dividing the scene in the scene coordinate system, and obtaining a plurality of grids; for each of the plurality of grids, allocating the first physical coordinate of the at least one object to a corresponding grid on the basis of the first physical coordinate of the at least one object in the scene coordinate system; fusing the first physical coordinates of the object contained in each grid, and obtaining a fused physical coordinate; according to the calibration coordinate for the scene view, the actual width and the actual height of the scene, and the fused physical coordinate, generating initial hotspot data based on an image coordinate system; and sending the initial hotspot data to the terminal, for the terminal to generate a heat map in the scene on the basis of the initial hotspot data.
Need to check novelty before this filing date? Find Prior Art

Description

Data processing method, heat map generation method and device Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method and device, a heat map generation method and device, an image processing method and device, an electronic device, a storage medium, and a computer program product. Background Art

[0002] Heat maps highlight the level of interest in various areas within a scene, clearly and intuitively displaying the characteristics of each area. In scenarios such as smart cities and smart retail, heat maps can be used to indicate the level of interest in each area, providing a basis for adjusting the scene's layout.

[0003] Summary of the Invention

[0004] The present disclosure provides a data processing method and apparatus, a heat map generation method and apparatus, an image processing method and apparatus, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a data processing method is provided, including: receiving first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal; the calibrated coordinates characterize the size of a view area corresponding to the scene in the scene view; dividing the scene in the scene coordinate system to obtain a plurality of grids; for each of the plurality of grids, based on the first physical coordinates of at least one object in the scene coordinate system, respectively assigning the first physical coordinates of at least one object to the corresponding grid; fusing the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; generating initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene, and the fused physical coordinates; and sending the initial hotspot data to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0006] According to an embodiment of the present disclosure, generating initial hotspot data based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene, and the fused physical coordinates includes: converting the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene; determining the initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and actual height of the scene; and determining the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0007] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in an image in a scene coordinate system include the first physical coordinates of the object contained in the images respectively captured at multiple image capture moments in the scene coordinate system; based on the first physical coordinates of at least one object in the scene coordinate system, respectively allocating the first physical coordinates of at least one object to corresponding grids includes: determining the number of first physical coordinates of the object contained in the images respectively captured at multiple image capture moments; and in response to determining that the number exceeds a preset threshold, allocating the first physical coordinates of the object contained in the image captured at each image capture moment to the corresponding grid, so as to obtain the first physical coordinates of the object contained in each grid at each image capture moment.

[0008] According to an embodiment of the present disclosure, the first physical coordinates of the objects contained in each grid are fused to obtain the fused physical coordinates, including: fusing the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and fusing the initial fused physical coordinates in each grid at multiple image acquisition moments to obtain the fused physical coordinates.

[0009] According to another aspect of the present disclosure, a heat map generation method is provided, including: sending calibrated coordinates for a scene view to a server; the calibrated coordinates represent the size of a view area corresponding to a scene in the scene view; in response to receiving initial hotspot data from the server, obtaining a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; based on the first height, the first width, and the original height and the original width, converting the initial hotspot data into target hotspot data at the first page resolution; and generating a heat map within the scene based on the target hotspot data.

[0010] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; based on the first height, the first width, and the original height and the original width, converting the initial hotspot data into target hotspot data at the first page resolution includes: determining a first width ratio based on the first width and the original width; determining a first height ratio based on the first height and the original height; converting the initial fused pixel coordinates into target pixel coordinates according to the first width ratio and the first height ratio; converting the initial display radius into a target display radius according to the first width ratio; and determining the target hotspot data according to the target pixel coordinates and the target display radius.

[0011] According to an embodiment of the present disclosure, the above-mentioned heat map generation method also includes: obtaining the second height and second width of the scene view at the second page resolution; receiving configuration information for the origin coordinates of the scene view; based on the origin coordinates of the scene view, using the second height and second width as well as the original height and original width, calibrating the view area in the scene view to obtain the calibrated coordinates for the scene view.

[0012] According to an embodiment of the present disclosure, the view area in the scene view is calibrated using the second height and the second width as well as the original height and the original width to obtain the calibrated coordinates for the scene view, including: receiving a calibration operation for the view area to determine calibrated position information for the view area; determining a second height ratio based on the second height and the original height; determining a second width ratio based on the second width and the original width; and determining the calibrated coordinates based on the second height ratio, the second width ratio, and the calibrated position information for the view area.

[0013] According to another aspect of the present disclosure, an image processing method is provided, including: acquiring an image captured at each image capture moment; determining the pixel coordinates of at least one object in the image in an image coordinate system; converting the pixel coordinates of the at least one object in the image coordinate system into a second physical coordinate in a scene coordinate system relative to the origin coordinates of the image coordinate system; correcting the second physical coordinates based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain a first physical coordinate of the at least one object in the scene coordinate system; and sending the first physical coordinates of the at least one object in the scene coordinate system to a server.

[0014] According to an embodiment of the present disclosure, based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, the second physical coordinates are corrected to obtain the first physical coordinates of at least one object in the scene coordinate system, including: based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, determining the coordinate correction value of the origin coordinates of the image coordinate system in the scene coordinate system relative to the origin coordinates of the scene coordinate system; and correcting the second physical coordinates based on the coordinate correction value to obtain the first physical coordinates of at least one object in the scene coordinate system.

[0015] According to another aspect of the present disclosure, a data processing device is provided, including: a first receiving module for receiving the first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and the calibrated coordinates for the scene view from a terminal; the calibrated coordinates represent the size of the view area corresponding to the scene in the scene view; a division module for dividing the scene in the scene coordinate system to obtain a plurality of grids; an allocation module for allocating the first physical coordinates of at least one object to the corresponding grid for each of the plurality of grids based on the first physical coordinates of at least one object in the scene coordinate system; a fusion module for fusing the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; a first generation module for generating initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene and the fused physical coordinates; and a first sending module for sending the initial hotspot data to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0016] According to an embodiment of the present disclosure, the first generation module includes: a first conversion unit, used to convert the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; a first determination unit, used to determine the initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and a second determination unit, used to determine the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0017] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively captured at multiple image capture moments in the scene coordinate system; the allocation module includes: a third determination unit, used to determine the number of first physical coordinates of the object contained in the images respectively captured at multiple image capture moments; and an allocation unit, used to allocate the first physical coordinates of the object contained in the image captured at each image capture moment to the corresponding grid in response to the determination that the number exceeds a preset threshold, so as to obtain the first physical coordinates of the object contained in each grid at each image capture moment.

[0018] According to an embodiment of the present disclosure, the fusion module includes: a first fusion unit, which is used to fuse the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and a second fusion unit, which is used to fuse the initial fused physical coordinates in each grid at multiple image acquisition moments to obtain fused physical coordinates.

[0019] According to another aspect of the present disclosure, a heat map generation device is provided, including: a second sending module for sending calibrated coordinates for a scene view to a server; the calibrated coordinates represent the size of a view area corresponding to a scene in the scene view; a first acquisition module for acquiring, in response to receiving initial hotspot data from the server, a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; a first conversion module for converting the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, and the original height and the original width; and a second generation module for generating a heat map within the scene based on the target hotspot data.

[0020] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; the first conversion module includes: a fourth determination unit for determining a first width ratio based on the first width and the original width; a fifth determination unit for determining a first height ratio based on the first height and the original height; a second conversion unit for converting the initial fused pixel coordinates into target pixel coordinates according to the first width ratio and the first height ratio; a third conversion unit for converting the initial display radius into a target display radius according to the first width ratio; and a sixth determination unit for determining the target hotspot data based on the target pixel coordinates and the target display radius.

[0021] According to an embodiment of the present disclosure, the heat map generation device also includes: a second acquisition module, used to obtain the second height and second width of the scene view at the second page resolution; a second receiving module, used to receive configuration information of the origin coordinates of the scene view; a calibration module, used to calibrate the view area in the scene view based on the origin coordinates of the scene view, using the second height and second width and the original height and original width, to obtain the calibrated coordinates for the scene view.

[0022] According to an embodiment of the present disclosure, the calibration module includes: a receiving unit for receiving a calibration operation for a view area to determine calibration position information for the view area; a seventh determination unit for determining a second height ratio based on the second height and the original height; an eighth determination unit for determining a second width ratio based on the second width and the original width; and a ninth determination unit for determining calibration coordinates based on the second height ratio, the second width ratio and the calibration position information for the view area.

[0023] According to another aspect of the present disclosure, an image processing device is provided, including: a third acquisition module for acquiring an image captured at each image capture moment; a determination module for determining the pixel coordinates of at least one object in the image in an image coordinate system; a second conversion module for converting the pixel coordinates of at least one object in the image coordinate system into a second physical coordinate in a scene coordinate system relative to the origin coordinates of the image coordinate system; a correction module for correcting the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of the at least one object in the scene coordinate system; and a third sending module for sending the first physical coordinates of the at least one object in the scene coordinate system to a server.

[0024] According to an embodiment of the present disclosure, the correction module includes: a tenth determination unit, which is used to determine the coordinate correction value of the origin coordinates of the image coordinate system in the scene coordinate system relative to the origin coordinates of the scene coordinate system based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system; and a correction unit, which is used to correct the second physical coordinate based on the coordinate correction value to obtain the first physical coordinate of at least one object in the scene coordinate system.

[0025] According to another aspect of the present disclosure, an electronic device is provided, including: a memory and a processor, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes the method described above.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute and implement the method described above.

[0027] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0028] According to another aspect of the present disclosure, a cloud server is provided, comprising the data processing device as described above.

[0029] According to another aspect of the present disclosure, a terminal is provided, comprising the heat map generating device as described above.

[0030] According to another aspect of the present disclosure, an image acquisition device is provided, comprising the image processing apparatus as described above.

[0031] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0033] FIG1 is a system architecture applicable to a data processing method and apparatus, a heat map generation method and apparatus, and an image processing method and apparatus according to an embodiment of the present disclosure;

[0034] FIG2 is a flow chart of a data processing method according to an embodiment of the present disclosure;

[0035] FIG3 is a schematic diagram of determining the physical coordinates of each grid according to an embodiment of the present disclosure;

[0036] FIG4 is a flow chart of a method for acquiring fused physical coordinates according to an embodiment of the present disclosure;

[0037] FIG5 is a flow chart of a method for generating a heat map according to an embodiment of the present disclosure;

[0038] FIG6 is a schematic diagram of a process for determining calibration coordinates for a scene view according to an embodiment of the present disclosure;

[0039] FIG7A and FIG7B are respectively the renderings of heat maps generated at different page resolutions;

[0040] FIG8 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0041] FIG9 schematically shows second physical coordinates of multiple objects in an image in a scene coordinate system relative to the origin coordinates of an image coordinate system;

[0042] FIG10 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0043] FIG11 is a block diagram of a heat map generating apparatus according to an embodiment of the present disclosure;

[0044] FIG12 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0045] FIG13 is a block diagram of an electronic device for implementing the data processing method, the heat map generating method, and the image processing method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0046] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0047] It should be noted that the sequence numbers of the operations in the following method are only used to indicate the operation for the purpose of description and should not be regarded as indicating the order in which the operations should be performed. Unless explicitly stated, the method does not need to be performed in the order shown.

[0048] Figure 1 illustrates a system architecture applicable to a data processing method and apparatus, a heat map generation method and apparatus, and an image processing method and apparatus according to an embodiment of the present disclosure. It should be noted that Figure 1 merely illustrates an example of a system architecture applicable to embodiments of the present disclosure, to help those skilled in the art understand the technical content of the present disclosure, and does not imply that embodiments of the present disclosure cannot be applied in other environments or scenarios.

[0049] As shown in FIG1 , the system architecture 100 may include: an image acquisition device 101 , a cloud server 102 and a terminal 103 .

[0050] In the embodiment of the present disclosure, the image acquisition device 101 can realize image acquisition and data processing functions. The image acquisition device 101 can include multiple image acquisition devices, or be connected to multiple image acquisition devices through communication.

[0051] In the embodiment of the present disclosure, the cloud server 102 can implement data processing functions.

[0052] In the embodiment of the present disclosure, the terminal 103 can be used to implement image display and data processing functions. In some embodiments, the terminal 103 can also be connected to an external display device, and the content processed by the terminal 103 can be displayed on the external display device.

[0053] In an embodiment of the present disclosure, the image acquisition device 101 may acquire an image captured at each image acquisition moment and, based on the correspondence between the image coordinate system and the scene coordinate system, convert the pixel coordinates of at least one object in the image into first physical coordinates in the scene coordinate system. The first physical coordinates of the at least one object in the scene coordinate system are then transmitted to the cloud server 102. The terminal 103 may calibrate the viewing area corresponding to the scene in the scene view and transmit the calibrated coordinates of the calibrated scene view to the cloud server 102. After receiving the first physical coordinates of the at least one object in the scene coordinate system and the calibrated coordinates for the scene view, the cloud server 102 may generate initial hotspot data based on the first physical coordinates and calibrated coordinates of each object and transmit the initial hotspot data to the terminal 103. After receiving the initial hotspot data returned by the cloud server 102, the terminal 103 may also convert the initial hotspot data to obtain target hotspot data that matches the current page resolution. A heat map of the scene is then generated based on the target hotspot data and displayed.

[0054] It should be understood that the number of image acquisition devices 101 and cloud servers 102 in FIG1 is merely illustrative and any number of image acquisition devices 101 and cloud servers 102 may be provided according to implementation requirements.

[0055] It should be noted that the sequence numbers of the operations in the following method are only used to indicate the operation for the purpose of description, and should not be regarded as indicating the order in which the operations should be performed. Unless explicitly stated, the method does not need to be performed in the order shown.

[0056] FIG2 is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0057] As shown in Figure 2, the data processing method 200 may include operations S210 to S260. In the embodiment of the present disclosure, the data processing method 200 may be executed by a server (eg, a cloud server) to perform the relevant operations in the embodiment of the present disclosure.

[0058] In operation S210 , first physical coordinates of at least one object in an image in a scene coordinate system are received from an image acquisition device and calibration coordinates for a scene view are received from a terminal.

[0059] In operation S220 , the scene is divided in the scene coordinate system to obtain a plurality of grids.

[0060] In operation S230 , for each of the plurality of grids, first physical coordinates of the at least one object are respectively allocated to a corresponding grid based on the first physical coordinates of the at least one object in the scene coordinate system.

[0061] In operation S240 , a fusion process is performed on the first physical coordinates of the objects included in the respective grids to obtain fused physical coordinates.

[0062] In operation S250 , initial hotspot data based on the image coordinate system is generated according to the calibrated coordinates for the scene view, the actual width and height of the scene, and the fused physical coordinates.

[0063] In operation S260 , the initial hotspot data is sent to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0064] According to an embodiment of the present disclosure, a server (eg, a cloud server) may receive first physical coordinates of at least one object in an image sent by an image acquisition device in a scene coordinate system and calibration coordinates for a scene view from a terminal.

[0065] In the disclosed embodiments, the first physical coordinates of at least one object in an image in the scene coordinate system are obtained by converting the pixel coordinates of the at least one object in the image in the image coordinate system, based on images of the object in the scene at corresponding moments captured by an image acquisition device at regular time intervals. The time interval can be set as needed and is not limited herein.

[0066] In the embodiment of the present disclosure, the calibrated coordinates of the scene view can be used to represent the size of the view area corresponding to the scene in the scene view. The area covered by the view area is the scene area where the objects of interest to each object are located.

[0067] It is understandable that the scene view may contain noise areas other than the view area. The noise area may be an area where object statistics do not need to be performed, and the area can be set according to actual conditions. In addition, the noise area may also refer to an area that is meaningless to object statistics. For example, when the terminal is used to display the view area, due to changes in page resolution or page zooming, some noise areas appear around the view area, and the area covered by the noise area does not include the scene area where the object of interest is located. In view of this, the actual size of the view area in the scene view can be calibrated by obtaining the calibration coordinates for the scene view, thereby avoiding interference from the noise area and facilitating subsequent processing.

[0068] FIG3 is a schematic diagram of determining the physical coordinates of each grid according to an embodiment of the present disclosure. An example process of determining the physical coordinates of each grid is described below with reference to FIG3 .

[0069] As shown in FIG3 , in the scene coordinate system, the actual width X and actual height Y of scene 1330 can be determined based on the location information of scene 1330. Scene 1330 is then divided into multiple grids according to a preset grid size grid_length. The grid size can be set based on actual needs, for example, a grid size of 0.1 m, 0.2 m, etc.

[0070] For example, determine the horizontal number X_totle of grids obtained by dividing the actual width X of the scene 1330, and determine the vertical number Y_totle of grids obtained by dividing the actual height Y of the scene 1330. X_totle and Y_totle are determined using the following formulas (1) and (2), respectively. X_totle = int(math.ceil(X / grid_length)) (1) Y_totle = int(math.ceil(Y / grid_length)) (2)

[0071] Here, math.ceil indicates rounding up to an integer, that is, if X / grid_length is an integer, then X_totle is equal to (X / grid_length), otherwise X_totle=X / grid_length+1.

[0072] The scene 1330 is segmented based on X_totle and Y_totle to obtain multiple grids, each of which is the minimum rectangle in Figure 3 (for example only). After obtaining multiple grids, the physical coordinates of each grid can be determined. For example, taking the physical coordinates of a grid as an example, the physical coordinates of the grid include four vertex coordinates, which are the upper left vertex coordinate W1, the upper right vertex coordinate W2, the lower right vertex coordinate W3, and the lower left vertex coordinate W4. The coordinates of each vertex are shown in formulas (3) to (6). W1 = [W1_x, W1_y] = [W_left_top_x + x_row*grid_length, W_left_top_y + y_column*grid_length] (3) W2 = [W1_x + grid_length, W1_y] (4)

[0073] W3=[W1_x+grid_length, W1_y+grid_length] (5)

[0074] W4=[W1_x, W1_y+grid_length] (6)

[0075] In formulas (3) to (6), W_left_top_x and W_left_top_y are the horizontal and vertical coordinates of the top left vertex of the scene, respectively. The grids are labeled a, b, c, d, e, and f from top to bottom and left to right. Grid a has x_row = 0 and y_column = 0, grid b has x_row = 0 and y_column = 1, and grid e has x_row = 1 and y_column = 0.

[0076] It can be understood that calculating the physical coordinates of each grid based on the location information of the scene area, the size of the grid, and the identification of the grid can improve the calculation efficiency and accuracy of the physical coordinates of the grid.

[0077] Next, for each of the plurality of grids, based on the first physical coordinates of the at least one object in the scene coordinate system, the first physical coordinates of the at least one object are respectively allocated to the corresponding grid.

[0078] Afterwards, the first physical coordinates of the objects contained in each grid are fused to obtain fused physical coordinates.

[0079] For example, suppose the i-th grid contains the first physical coordinates of m objects, i = 1, 2, ..., X_totle * Y_totle, m is a positive integer. Determine the average value of the horizontal coordinates of the m first physical coordinates in the i-th grid and the average value of the vertical coordinates of the m first physical coordinates and The following formulas (7) and (8) are used to determine respectively.

[0080] In formulas (7) to (8), x j and y j They respectively represent the abscissa and ordinate of the j-th (j=1, 2, ..., m) first physical coordinate.

[0081] Sure and Afterwards, according to and The fused physical coordinates can be obtained

[0082] Next, based on the correspondence between the scene coordinate system and the image coordinate system, the fused physical coordinates in the scene coordinate system are converted into initial hotspot data in the image coordinate system using the calibrated coordinates for the scene view, the actual width and height of the scene, and the fused physical coordinates. This initial hotspot data is then sent to the terminal, which then generates a heat map of the scene based on the initial hotspot data.

[0083] In the technical solution disclosed herein, on the one hand, the actual size of the viewing area in the scene view is calibrated by obtaining calibration coordinates for the scene view to avoid interference from noise areas, thereby providing a basis for improving the accuracy of the initial hotspot data. On the other hand, based on the correspondence between the scene coordinate system and the image coordinate system, the present disclosure converts physical coordinates into pixel coordinates, thereby ensuring the accuracy of the initial hotspot data and, in turn, the accuracy of the heat map.

[0084] As previously mentioned, the first physical coordinates are obtained by converting the pixel coordinates of at least one object in the image coordinate system into the image coordinate system. The server then performs fusion processing based on the first physical coordinates of the at least one object to obtain the fused physical coordinates.

[0085] In some embodiments, the first physical coordinates of at least one object in an image in the scene coordinate system may include the first physical coordinates of the object in the scene coordinate system contained in images captured at multiple image capture times. If the server receives too many first physical coordinates, the server's storage and computing capabilities may be exceeded, resulting in data overflow, thereby affecting the normal generation of initial hotspot data.

[0086] In the disclosed embodiment, the first physical coordinates obtained at multiple image capture moments can be segmented and processed according to time periods. This allows the server to fuse only the first physical coordinates corresponding to each time period to obtain the initial fused physical coordinates. The aggregated initial physical coordinates from each segment are then fused to obtain the fused physical coordinates. This reduces the number of first physical coordinates that the server must process each time, thereby resolving issues such as memory overflow and the inability to properly generate initial hotspot data.

[0087] The process of segmentation processing is described below with reference to FIG4 .

[0088] FIG4 is a flowchart of a method for acquiring fused physical coordinates according to an embodiment of the present disclosure.

[0089] As shown in FIG. 4 , the method for acquiring fused physical coordinates may include operations S331 to S334 and operations S341 to S343 .

[0090] In operation S331 , the number of first physical coordinates of objects contained in images respectively captured at a plurality of image capturing moments is determined.

[0091] In operation S332, it is determined whether the number of first physical coordinates of objects contained in the images captured at the multiple image capture moments exceeds a preset threshold. If it is determined that the number exceeds the preset threshold, operations S333 and S341-S342 are performed; otherwise, operations S334 and S343 are performed.

[0092] In operation S333 , the first physical coordinates of the objects contained in the images captured at each image capturing moment are allocated to corresponding grids to obtain the first physical coordinates of the objects contained in each grid at each image capturing moment.

[0093] In operation S334 , the first physical coordinates of the objects contained in the images captured at the multiple image capture moments are allocated to corresponding grids to obtain the first physical coordinates of the objects contained in each grid.

[0094] In operation S341 , the first physical coordinates of the objects contained in each grid at each image acquisition moment are fused to obtain the initial fused physical coordinates of each grid at each image acquisition moment.

[0095] In operation S342 , the initial fused physical coordinates in each network at the multiple image acquisition moments are fused to obtain fused physical coordinates.

[0096] In operation S343 , a fusion process is performed on the first physical coordinates of the objects included in the respective grids to obtain fused physical coordinates.

[0097] According to an embodiment of the present disclosure, if it is determined that the number of first physical coordinates of objects contained in images captured at multiple image capture moments exceeds a preset threshold, it means that the number of first physical coordinates received by the server exceeds the processing capacity of the server. At this time, the first physical coordinates obtained at multiple image capture moments can be segmented according to time periods to reduce the number of first physical coordinates processed by the server each time. If it is determined that the number of first physical coordinates of objects contained in images captured at multiple image capture moments does not exceed the preset threshold, it means that the storage capacity and computing capacity of the server can meet the processing requirements for the first physical coordinates. At this time, the first physical coordinates corresponding to the multiple image capture moments can be directly fused to obtain fused physical coordinates.

[0098] It should be noted that, in the disclosed embodiments, the preset threshold value can be set according to actual conditions and is not specifically limited.

[0099] In addition, when segmenting the first physical coordinates obtained at multiple image acquisition moments, the first physical coordinates may also be segmented according to preset time periods based on actual conditions, which is not limited here.

[0100] According to an embodiment of the present disclosure, the above-mentioned initial hotspot data may be generated in the following manner.

[0101] For example, the fused physical coordinates may be converted to initial fused pixel coordinates based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width, and the actual height of the scene.

[0102] For the sake of simplicity, the following description is made by taking the view area in the scene view as a regular rectangular area as an example. It should be understood that the solution disclosed herein is not limited to this.

[0103] Assume that the calibration coordinates for the scene view include two vertex coordinates, which are the upper left vertex coordinates and the lower right vertex coordinates corresponding to the view area. Taking the upper left vertex of the scene view as the origin coordinate, the upper left vertex coordinates of the view area can be expressed as (left_top_x, left_top_y), and the lower right vertex coordinates can be expressed as (right_bottom_x, right_bottom_y).

[0104] For example, the following formulas (9) to (10) may be used to determine the initial fused pixel coordinates.

[0105] In formulas (9) to (10), pixel_x and pixel_y are the horizontal and vertical coordinates of the initial fused pixel coordinates, respectively. are the horizontal and vertical coordinates of the fused physical coordinates, and X and Y are the actual width and height of the scene respectively.

[0106] In addition, the initial display radius of the fused pixel coordinates may be determined based on the calibrated coordinates for the scene view, the actual width, and the actual height of the scene.

[0107] For example, the following formula (11) can be used to determine the initial display radius: pixel_radius = Math.max((right_bottom_x-lefi_top_x) / X, (right_bottom_y-lefi_top_y) / Y) (11)

[0108] In formula (11), pixel_radius represents the initial display radius.

[0109] Afterwards, the initial fused pixel coordinates and the initial display radius are determined as initial hotspot data.

[0110] In the embodiments of the present disclosure, for example, the corresponding relationships described in the above formulas (9) to (11) can be used to convert the physical coordinates into pixel coordinates based on the calibrated coordinates of the scene view, the actual width and actual height of the scene, and the fused physical coordinates, thereby ensuring the accuracy of the initial hotspot data and further ensuring the accuracy of the heat map.

[0111] FIG5 is a flowchart of a method for generating a heat map according to an embodiment of the present disclosure.

[0112] As shown in Figure 5, the heat map generation method 400 may include operations S410 to S440. In the embodiment of the present disclosure, the heat map generation method 400 may be performed by a terminal as an execution subject to perform the relevant operations in the embodiment of the present disclosure.

[0113] In operation S410 , the calibration coordinates for the scene view are sent to the server.

[0114] In operation S420 , in response to receiving the initial hotspot data from the server, a first height and a first width of the scene view at the first page resolution and an original height and an original width of the scene view at the original page resolution are obtained.

[0115] In operation S430 , the initial hotspot data is converted into target hotspot data at a first page resolution based on the first height, the first width, and the original height and width.

[0116] In operation S440 , a heat map within the scene is generated based on the target hotspot data.

[0117] According to an embodiment of the present disclosure, the terminal can calibrate the actual size of the view area corresponding to the scene in the scene view at a second page resolution to obtain calibrated coordinates for the scene view. The second page resolution is used to indicate the page resolution corresponding to the scene view when calibrating the size of the scene view.

[0118] After determining the calibration coordinates for the scene view, the terminal can send the calibration coordinates to the server (such as a cloud server) so that the server can convert the physical coordinates to pixel coordinates based on the calibration coordinates and generate initial hotspot data based on the image coordinate system.

[0119] It is understood that when the terminal generates a heat map, when the page resolution changes, it will affect the display effect of the hotspot data on the heat map. For example, the change of page resolution will cause the hotspot data to shift, resulting in the hotspot data being incompatible with the current page resolution.

[0120] In order to solve the offset of hotspot data caused by different page resolutions, in an embodiment of the present disclosure, after receiving the initial hotspot data from the server, the terminal can obtain the first height and first width of the scene view at the first page resolution and the original height and original width of the scene view at the original page resolution. Among them, the first page resolution is used to indicate the page resolution corresponding to the scene view when the heat map is to be generated, that is, the current page resolution. The original page resolution is used to indicate the page resolution corresponding to the scene view before the scene view is calibrated.

[0121] Afterwards, the terminal can convert the initial hotspot data into target hotspot data at the first page resolution based on the first height, the first width, and the original height and width. Then, based on the target hotspot data, a heat map within the scene is generated. In this way, the initial hotspot data can be converted into target hotspot data that matches the current page resolution, which is conducive to displaying a page adapted to the current page resolution, thereby ensuring the accuracy of the heat map.

[0122] According to an embodiment of the present disclosure, the calibration coordinates for the scene view may be determined in the following manner.

[0123] It should be noted that, for simplicity of explanation, the following uses the view area in the scene view as a regular rectangular area as an example, and refers to Figure 6 to illustrate the process of determining the calibration coordinates for the scene view. It should be understood that the content shown in Figure 6 is merely exemplary and is intended to help understand the solution of the present disclosure, but the present disclosure is not limited to this.

[0124] FIG6 is a schematic diagram of a process of determining calibration coordinates for a scene view according to an embodiment of the present disclosure.

[0125] After the terminal acquires the scene view, the display scale of the scene view can be adjusted on the terminal's display interface to achieve a good display effect. As shown in Figure 6, the scaled scene view 510 includes a view area 520 corresponding to the scene and a noise area outside of the view area 520 (e.g., the area between the scene view 510 and the view area 520). The area covered by the noise area does not include the scene area where the object of interest is located.

[0126] In order to avoid interference from the noise area, in the embodiment of the present disclosure, the actual size of the viewing area 520 may be calibrated.

[0127] For example, the second height scaling_height and the second width scaling_width of the scene view 510 at the second page resolution may be obtained, wherein the second page resolution refers to the page resolution corresponding to the scene view after the display ratio of the scene view is scaled and adjusted.

[0128] Afterwards, configuration information of the origin coordinates of the scene view 510 is received. For example, according to the configuration information, it is determined that the origin coordinates of the scene view 510 are located at the upper left vertex of the scene view 510, that is, point O.

[0129] Next, based on the origin coordinates of the scene view 510 , the view area 520 in the scene view 510 may be calibrated using the second height and the second width as well as the original height and the original width to obtain calibrated coordinates for the scene view 510 .

[0130] For example, a calibration operation for view area 520 is received to determine calibration position information for view area 520. For example, based on the calibration operation for view area 520, multiple calibration points in view area 520 may be determined, such as calibration point 521, calibration point 522, calibration point 523, and calibration point 524. The multiple calibration points may be used to calibrate the actual size of view area 520.

[0131] Each calibration point corresponds to a calibration position information, which includes the width click_x and height click_y corresponding to the calibration point. The width click_x and height click_y refer to the width and height relative to the position of point O, respectively.

[0132] Next, a second height ratio may be determined based on the second height and the original height. A second width ratio may be determined based on the second width and the original width. Then, calibration coordinates may be determined based on the second height ratio, the second width ratio, and the calibration position information for the view area.

[0133] For example, the following formulas (12) and (13) can be used to determine the calibration coordinates of each calibration point: result_x = click_x*(resolution_width / scaling_width) (12) result_y = click_y*(resolution_height / scaling_height) (13)

[0134] In the above formulas (12) to (13), resolution_width and resolution_height respectively represent the original width and original height of the scene view at the original page resolution, scaling_width and scaling_height respectively represent the second width and second height of the scene view at the second page resolution, click_x and click_y represent the calibration position information of each calibration point, and result_x and result_y respectively represent the horizontal coordinate and vertical coordinate of the calibration coordinate of each calibration point.

[0135] Based on the above formulas (12) and (13), the calibration coordinates corresponding to calibration points 521 to 524 can be determined. Based on the calibration coordinates corresponding to calibration points 521 to 524, the size of view area 520 can be determined. In this way, the size of view area 520 is calibrated.

[0136] In some embodiments, the size of the view area 520 may be calibrated according to the calibration coordinates corresponding to the calibration points 521 and 523 or the calibration coordinates corresponding to the calibration points 522 and 524, and the specific size may be selected according to actual needs.

[0137] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius. Based on the first height, the first width, and the original height and width, converting the initial hotspot data into target hotspot data at the first page resolution may include the following operations.

[0138] For example, a first width ratio is determined based on the first width and the original width. A first height ratio is determined based on the first height and the original height. The initial fused pixel coordinates are then converted to target pixel coordinates based on the first width ratio and the first height ratio. The initial display radius is then converted to a target display radius based on the first width ratio. Target hotspot data is then determined based on the target pixel coordinates and the target display radius.

[0139] For example, the target pixel coordinates and target display radius can be determined using the following formulas (14) to (16): current_pixel_x = pixel_x * current_resolution_width / resolution_width (14) current_pixel_y = pixel_y * current_resolution_height / resolution_height (15) current_pixel_radius = pixel_radius * current_resolution_width / resolution_width (16)

[0140] In formulas (14) to (16), current_pixel_x and current_pixel_y represent the horizontal and vertical coordinates of the target pixel coordinates, respectively, current_pixel_radius represents the target display radius, pixel_x and pixel_y represent the horizontal and vertical coordinates of the initial fused pixel coordinates, respectively, pixel_radius represents the initial display radius, resolution_width and resolution_height represent the original width and original height of the scene view at the original page resolution, respectively, and current_resolution_width and current_resolution_height represent the first width and first height of the scene view at the first page resolution, respectively.

[0141] Based on the above formulas (14) to (16), the initial hotspot data can be converted into target hotspot data that matches the current page resolution, thereby achieving page display adapted to different page resolutions.

[0142] FIG7A and FIG7B are respectively diagrams showing the effects of heat maps generated at different page resolutions.

[0143] Figure 7A is a heat map generated at a page resolution of 854*480, and Figure 7B is a heat map generated at a page resolution of 1916*1078. As shown in Figures 7A and 7B, the heat map generated using the solution of the present disclosure can be adapted to pages with different page resolutions for display. In addition, based on the heat map, the degree of attention paid to each area in the scene by the object can be clearly and intuitively displayed. Among them, the light spots shown in the figure represent objects. If the number of light spots in a certain area is greater, the corresponding light spot has a darker color, indicating that the area is receiving more attention.

[0144] FIG8 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0145] As shown in Figure 8, the image processing method 700 may include operations S710 to S750. In the embodiment of the present disclosure, the image processing method 700 may be executed by an image acquisition device as an execution subject to perform the relevant operations in the embodiment of the present disclosure.

[0146] In operation S710 , an image captured at each image capturing moment is acquired.

[0147] In operation S720 , pixel coordinates of at least one object in the image in the image coordinate system are determined.

[0148] In operation S730, pixel coordinates of the at least one object in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0149] In operation S740, the second physical coordinates are corrected based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain first physical coordinates of the at least one object in the scene coordinate system.

[0150] In operation S750 , first physical coordinates of at least one object in the scene coordinate system are sent to the server.

[0151] According to an embodiment of the present disclosure, an image acquisition device may include multiple image acquisition devices. The multiple image acquisition devices (e.g., cameras) are arranged at corresponding positions in a scene. The multiple image acquisition devices may cover the area where the scene is located.

[0152] The image acquisition device can acquire images captured by multiple image acquisition devices at each image acquisition moment. The image acquisition moment can be set based on actual circumstances and is not limited here. After acquiring the image captured at each image acquisition moment, the image acquisition device can determine the pixel coordinates of at least one object in the image in the image coordinate system.

[0153] Then, based on the correspondence between the image coordinate system and the scene coordinate system, the pixel coordinates of the at least one object in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0154] Next, based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, the second physical coordinates are corrected to obtain the first physical coordinates of the at least one object in the scene coordinate system. It should be noted that the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system can be the same point or different points, depending on the actual scene setting.

[0155] Then, the first physical coordinates of the at least one object in the scene coordinate system are sent to a server (eg, a cloud server) so that the server generates initial hotspot data based on the first physical coordinates.

[0156] According to an embodiment of the present disclosure, the pixel coordinates of at least one object in the image are converted and corrected by utilizing an image acquisition device to obtain the first physical coordinates of each object in the scene coordinate system, and the first physical coordinates of each object in the scene coordinate system are sent to the server. This is conducive to real-time processing of the image to obtain corresponding hotspot data, thereby facilitating rapid acquisition of the density distribution of objects in the scene.

[0157] According to an embodiment of the present disclosure, the correspondence between the image coordinate system and the scene coordinate system may be determined in the following manner.

[0158] For example, four marking points may be determined in the image, and pixel coordinates of each of the four marking points in the image coordinate system and third physical coordinates of the four marking points relative to the origin coordinates of the image coordinate system may be obtained.

[0159] The pixel coordinates in the image coordinate system are converted to physical coordinates in the scene coordinate system (world coordinate system) using the following formula (17).

[0160] In formula (17), (x w ,y w , z w ) represents the physical coordinates in the scene coordinate system, (x im ,y im ) represents the physical coordinate (x w ,y w , z w ) The pixel coordinates in the image coordinate system obtained by conversion, represents the affine matrix, a 11 ~a 32Represents the parameters in the affine matrix. In which, let the image plane be 1, and the coordinate has only one scale factor, so z im =1,a 33 =1.

[0161] According to formula (17), the following formulas (18) to (20) can be obtained. w =a 11 *x im +a 12 *y im +a 13 (18) y w =a 21 *x im +a 22 *y im +a 23 (19) z w =a 31 *x im +a 32 *y im +1 (20)

[0162] If you want to get z w Physical coordinates in the plane (x world ,y world ), the following formula (21) can be obtained by the projected triangle similarity method.

[0163] According to formulas (18) to (21), the physical coordinates (x world ,y world ).x world and y world It can be expressed as follows.

[0164] According to the pixel coordinates of the four markers in the image coordinate system, the third physical coordinates of the four markers relative to the origin of the image coordinate system, and formulas (22) to (23), the parameter a in the affine matrix can be determined: 11 ~a 32 .

[0165] The above parameter a can be determined using the following formula (24): 11 ~a 32 .

[0166] Determine the above parameter a 11 ~a 32Then, according to formulas (22) and (23), the correspondence between the image coordinate system and the scene coordinate system can be determined. Based on this correspondence, i.e., formulas (22) and (23), the pixel coordinates of at least one object in the image coordinate system can be converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0167] Fig. 9 schematically shows the second physical coordinates of multiple objects in an image in a scene coordinate system relative to the origin coordinates of an image coordinate system. The process of correcting the second physical coordinates is described below with reference to Fig. 9 .

[0168] Figure 9 shows two coordinate systems. The coordinate system with origin coordinates at point O1 is the scene coordinate system (xO1y), and the coordinate system with origin coordinates at point O2 is the image coordinate system (uO2v). Origin coordinates O1 and O2 can be the same point (i.e., O1 and O2 coincide), or different points (i.e., O1 and O2 do not coincide), without limitation. For ease of explanation, the following description uses the example of origin coordinates O1 and O2 being different points.

[0169] Assume that the image captured at each image capture moment includes four objects, for example, object A, object B, object C, and object D. Then, determine the pixel coordinates of each of these four objects in the image coordinate system. For example, the pixel coordinates corresponding to objects A to D are A(x1im, y1im), B(x2im, y2im), C(x3im, y3im), and D(x4im, y4im), respectively.

[0170] Next, based on the above formulas (22) and (23), the pixel coordinates of the four objects in the image coordinate system are converted into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system. As shown in Figure 9, the second physical coordinates of object A relative to the origin coordinates O2 of the image coordinate system are (x1, y1), the second physical coordinates of object B relative to the origin coordinates O2 of the image coordinate system are (x2, y2), the second physical coordinates of object C relative to the origin coordinates O2 of the image coordinate system are (x3, y3), and the second physical coordinates of object D relative to the origin coordinates O2 of the image coordinate system are (x4, y4).

[0171] Since there is a coordinate error between the origin coordinate O2 of the image coordinate system and the origin coordinate O1 of the scene coordinate system, it is necessary to correct the second physical coordinates of each object so as to convert the second physical coordinates into first physical coordinates relative to the origin coordinate O1 of the scene coordinate system.

[0172] For example, based on the correspondence between the origin coordinates O2 of the image coordinate system and the origin coordinates O1 of the scene coordinate system, a coordinate correction value of the origin coordinates O2 of the image coordinate system relative to the origin coordinates O1 of the scene coordinate system can be determined in the scene coordinate system. For example, the coordinate correction value can be determined based on the difference in the horizontal coordinates and the difference in the vertical coordinates between the origin coordinates O2 and the origin coordinates O1.

[0173] Afterwards, the second physical coordinates are corrected based on the coordinate correction value to obtain the first physical coordinates of the at least one object in the scene coordinate system.

[0174] For example, assuming the coordinate correction value is (Δx, Δy). Taking object A as an example, the second physical coordinate (x1, y1) of object A can be corrected using the following formula (25) to obtain the first physical coordinate (x1', y1') of object A in the scene coordinate system. x1'=x1+Δx,y1'=y1+Δy (25)

[0175] Similarly, based on the above method, the first physical coordinates of each of objects B to D in the scene coordinate system can be obtained.

[0176] According to an embodiment of the present disclosure, the second physical coordinates are corrected based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates, thereby improving the accuracy of the first physical coordinates.

[0177] FIG10 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0178] As shown in FIG. 10 , the data processing device 900 includes: a first receiving module 910 , a dividing module 920 , an allocating module 930 , a merging module 940 , a first generating module 950 and a first sending module 960 .

[0179] The first receiving module 910 is used to receive the first physical coordinates of at least one object in the image from the image acquisition device in the scene coordinate system and the calibrated coordinates for the scene view from the terminal; the calibrated coordinates represent the size of the view area corresponding to the scene in the scene view.

[0180] The division module 920 is used to divide the scene in the scene coordinate system to obtain multiple grids.

[0181] The allocation module 930 is configured to allocate the first physical coordinates of at least one object to a corresponding grid in each of the plurality of grids based on the first physical coordinates of the at least one object in the scene coordinate system.

[0182] The fusion module 940 is used to perform fusion processing on the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates.

[0183] The first generating module 950 is used to generate initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and actual height of the scene, and the fused physical coordinates.

[0184] The first sending module 960 is used to send the initial hotspot data to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

[0185] According to an embodiment of the present disclosure, the first generation module 950 includes: a first conversion unit, a first determination unit, and a second determination unit. The first conversion unit is configured to convert the fused physical coordinates into initial fused pixel coordinates based on the image coordinate system based on the calibrated coordinates for the scene view, the actual width, and the actual height of the scene; the first determination unit is configured to determine the initial display radius of the fused pixel coordinates based on the calibrated coordinates for the scene view, the actual width, and the actual height of the scene; and the second determination unit is configured to determine the initial fused pixel coordinates and the initial display radius as initial hotspot data.

[0186] According to an embodiment of the present disclosure, the first physical coordinates of at least one object in an image in a scene coordinate system include the first physical coordinates of the object contained in images captured at multiple image capture moments in the scene coordinate system; the allocation module 930 includes: a third determination unit and an allocation unit. The third determination unit is configured to determine the number of first physical coordinates of the objects contained in the images captured at the multiple image capture moments; and the allocation unit is configured to, in response to determining that the number exceeds a preset threshold, allocate the first physical coordinates of the objects contained in the images captured at each image capture moment to corresponding grids, thereby obtaining the first physical coordinates of the objects contained in each grid at each image capture moment.

[0187] According to an embodiment of the present disclosure, the fusion module 940 includes: a first fusion unit and a second fusion unit. The first fusion unit is configured to fuse the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates of each grid at each image acquisition moment; and the second fusion unit is configured to fuse the initial fused physical coordinates of each grid at multiple image acquisition moments to obtain the fused physical coordinates.

[0188] FIG11 is a block diagram of a heat map generating apparatus according to an embodiment of the present disclosure.

[0189] As shown in FIG. 11 , the heat map generating device 1000 includes: a second sending module 1010 , a first acquiring module 1020 , a first converting module 1030 and a second generating module 1040 .

[0190] The second sending module 1010 is used to send the calibration coordinates for the scene view to the server; the calibration coordinates represent the size of the view area corresponding to the scene in the scene view.

[0191] The first acquisition module 1020 is used to obtain the first height and first width of the scene view at the first page resolution and the original height and original width of the scene view at the original page resolution in response to receiving the initial hotspot data from the server.

[0192] The first conversion module 1030 is configured to convert the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, and the original height and width.

[0193] The second generating module 1040 is used to generate a heat map within the scene based on the target hotspot data.

[0194] According to an embodiment of the present disclosure, the initial hotspot data includes initial fused pixel coordinates and an initial display radius; the first conversion module 1030 includes: a fourth determination unit, a fifth determination unit, a second conversion unit, a third conversion unit, and a sixth determination unit. The fourth determination unit is configured to determine a first width ratio based on the first width and the original width; the fifth determination unit is configured to determine a first height ratio based on the first height and the original height; the second conversion unit is configured to convert the initial fused pixel coordinates into target pixel coordinates based on the first width ratio and the first height ratio; the third conversion unit is configured to convert the initial display radius into a target display radius based on the first width ratio; and the sixth determination unit is configured to determine target hotspot data based on the target pixel coordinates and the target display radius.

[0195] According to an embodiment of the present disclosure, the heat map generation device 1000 further includes: a second acquisition module, a second receiving module, and a calibration module. The second acquisition module is configured to acquire a second height and a second width of the scene view at a second page resolution; the second receiving module is configured to receive configuration information for the origin coordinates of the scene view; and the calibration module is configured to calibrate the view area in the scene view based on the origin coordinates of the scene view using the second height and the second width as well as the original height and the original width to obtain calibrated coordinates for the scene view.

[0196] According to an embodiment of the present disclosure, a calibration module includes: a receiving unit, a seventh determining unit, an eighth determining unit, and a ninth determining unit. The receiving unit is configured to receive a calibration operation for a view area to determine calibration position information for the view area; the seventh determining unit is configured to determine a second height ratio based on the second height and the original height; the eighth determining unit is configured to determine a second width ratio based on the second width and the original width; and the ninth determining unit is configured to determine calibration coordinates based on the second height ratio, the second width ratio, and the calibration position information for the view area.

[0197] FIG. 12 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0198] As shown in FIG. 12 , the image processing apparatus 1100 includes a third acquisition module 1110 , a determination module 1120 , a second conversion module 1130 , a correction module 1140 and a third sending module 1150 .

[0199] The third acquisition module 1110 is used to acquire the image captured at each image capture moment.

[0200] The determination module 1120 is configured to determine pixel coordinates of at least one object in the image in the image coordinate system.

[0201] The second conversion module 1130 is configured to convert pixel coordinates of at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system.

[0202] The correction module 1140 is configured to correct the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of at least one object in the scene coordinate system.

[0203] The third sending module 1150 is configured to send the first physical coordinates of at least one object in the scene coordinate system to the server.

[0204] According to an embodiment of the present disclosure, the correction module 1140 includes: a tenth determining unit, configured to determine, based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, a coordinate correction value of the origin coordinates of the image coordinate system relative to the origin coordinates of the scene coordinate system; and a correction unit, configured to correct the second physical coordinates based on the coordinate correction value to obtain the first physical coordinates of the at least one object in the scene coordinate system.

[0205] It should be noted that the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each module / unit / sub-unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0206] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (for example, including but not limited to user personal information) shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0207] In the technical solution disclosed herein, authorization or consent from the data owner is obtained before obtaining or collecting relevant data.

[0208] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0209] FIG13 schematically shows a block diagram of an electronic device suitable for implementing the data processing method, the heat map generating method, and the image processing method according to an embodiment of the present disclosure.

[0210] As shown in Figure 13, the electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include an onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0211] The RAM 1203 stores various programs and data required for the operation of the electronic device 1200. The processor 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. The processor 1201 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 1202 and / or the RAM 1203. It should be noted that the programs may also be stored in one or more memories other than the ROM 1202 and the RAM 1203. The processor 1201 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0212] According to an embodiment of the present disclosure, electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to bus 1204. Electronic device 1200 may further include one or more of the following components connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1206 including a cathode ray tube (CRT), liquid crystal display (LCD), speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card or modem. Communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 1210 as needed, so that computer programs read from the removable media can be installed into storage section 1208 as needed.

[0213] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0214] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1202 and / or RAM 1203 described above and / or one or more memories other than ROM 1202 and RAM 1203.

[0215] Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to cause the computer system to implement the data processing method, heat map generation method, and image processing method provided in the embodiments of the present disclosure.

[0216] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the processor 1201 executes the computer program. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0217] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 1209, and / or installed from the removable medium 1211. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0218] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209 and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0219] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0220] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0221] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0222] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: Receiving first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; Dividing the scene in a scene coordinate system to obtain a plurality of grids; For each of the plurality of grids, based on the first physical coordinate of the at least one object in the scene coordinate system, respectively assigning the first physical coordinate of the at least one object to the corresponding grid; Performing fusion processing on the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; Generate initial hotspot data based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates; as well as The initial hotspot data is sent to the terminal so that the terminal generates a heat map within the scene based on the initial hotspot data.

2. The method according to claim 1, wherein: The generating of initial hotspot data based on the image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates includes: According to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, the fused physical coordinates are converted into initial fused pixel coordinates based on an image coordinate system; Determining an initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and The initial fused pixel coordinates and the initial display radius are determined as the initial hotspot data.

3. The method according to claim 1 or 2, wherein: The first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively acquired at a plurality of image acquisition moments in the scene coordinate system; The allocating the first physical coordinates of the at least one object to corresponding grids based on the first physical coordinates of the at least one object in the scene coordinate system comprises: Determine the number of first physical coordinates of objects contained in the images respectively acquired at the plurality of image acquisition moments; as well as In response to determining that the number exceeds a preset threshold, the first physical coordinates of the objects contained in the image captured at each image capture moment are allocated to corresponding grids to obtain the first physical coordinates of the objects contained in each grid at each image capture moment.

4. The method according to claim 3, wherein: The fusing process of the first physical coordinates of the objects contained in each grid to obtain the fused physical coordinates includes: Fusing the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates of each grid at each image acquisition moment; and The initial fused physical coordinates in each network at the time of multiple image acquisition are fused to obtain the fused physical coordinates.

5. A method for generating a heat map, comprising: Send the calibration coordinates for the scene view to the server; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; In response to receiving the initial hotspot data from the server, obtaining a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; Based on the first height, the first width, the original height, and the original width, converting the initial hotspot data into target hotspot data at a first page resolution; as well as Based on the target hotspot data, a heat map within the scene is generated.

6. The method according to claim 5, wherein: The initial hotspot data includes initial fusion pixel coordinates and an initial display radius; and converting the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, the original height, and the original width includes: Determining a first width ratio based on the first width and the original width; Determining a first height ratio based on the first height and the original height; Converting the initial fused pixel coordinates to the target pixel coordinates according to the first width ratio and the first height ratio; Converting the initial display radius into a target display radius according to the first width ratio; and The target hotspot data is determined according to the target pixel coordinates and the target display radius.

7. The method according to claim 5, further comprising: Get the second height and the second width of the scene view at the second page resolution; Receiving configuration information of origin coordinates for the scene view; Based on the origin coordinates of the scene view, the view area in the scene view is calibrated using the second height and the second width as well as the original height and the original width to obtain calibrated coordinates for the scene view.

8. The method according to claim 7, wherein: The step of calibrating the view area in the scene view by using the second height and the second width as well as the original height and the original width to obtain the calibrated coordinates for the scene view includes: receiving a calibration operation for the viewing area to determine calibration position information for the viewing area; Determining a second height ratio according to the second height and the original height; determining a second width ratio according to the second width and the original width; and The calibration coordinates are determined based on the second height ratio, the second width ratio, and calibration position information for the view area.

9. An image processing method, comprising: Obtain the image collected at each image collection moment; determining pixel coordinates of at least one object in the image in an image coordinate system; Converting pixel coordinates of the at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system; Based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, correcting the second physical coordinates to obtain the first physical coordinates of the at least one object in the scene coordinate system; and The first physical coordinates of the at least one object in the scene coordinate system are sent to the server.

10. The method according to claim 9, wherein: The correcting the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system to obtain the first physical coordinates of the at least one object in the scene coordinate system comprises: Determining, based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, a coordinate correction value of the origin coordinates of the image coordinate system relative to the origin coordinates of the scene coordinate system in the scene coordinate system; and The second physical coordinates are corrected based on the coordinate correction value to obtain the first physical coordinates of the at least one object in the scene coordinate system.

11. A data processing device, comprising: A first receiving module, configured to receive first physical coordinates of at least one object in an image from an image acquisition device in a scene coordinate system and calibrated coordinates for a scene view from a terminal; the calibrated coordinates represent a size of a view area corresponding to a scene in the scene view; A division module, used for dividing the scene in a scene coordinate system to obtain a plurality of grids; an allocation module, configured to allocate the first physical coordinate of the at least one object to a corresponding grid respectively based on the first physical coordinate of the at least one object in the scene coordinate system for each grid in the plurality of grids; A fusion module, used for fusing the first physical coordinates of the objects contained in each grid to obtain fused physical coordinates; A first generating module, configured to generate initial hotspot data based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene, and the fused physical coordinates; as well as The first sending module is used to send the initial hotspot data to the terminal so that the terminal generates a heat map in the scene based on the initial hotspot data.

12. The device according to claim 11, wherein The first generation module comprises: A first conversion unit, configured to convert the fused physical coordinates into initial fused pixel coordinates based on an image coordinate system according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; A first determining unit, configured to determine an initial display radius of the fused pixel coordinates according to the calibrated coordinates for the scene view, the actual width and the actual height of the scene; and The second determining unit is used to determine the initial fusion pixel coordinates and the initial display radius as the initial hotspot data.

13. The device according to claim 11 or 12, wherein: The first physical coordinates of at least one object in the image in the scene coordinate system include the first physical coordinates of the object contained in the images respectively acquired at a plurality of image acquisition moments in the scene coordinate system; The allocation module comprises: A third determining unit, configured to determine the number of first physical coordinates of objects contained in the images respectively acquired at the plurality of image acquisition moments; as well as An allocating unit is used to allocate the first physical coordinates of the objects contained in the images collected at each image collection moment to the corresponding grids in response to determining that the number exceeds a preset threshold, so as to obtain the first physical coordinates of the objects contained in each grid at each image collection moment.

14. The device according to claim 13, wherein: The fusion module includes: A first fusion unit is used to fuse the first physical coordinates of the objects contained in each grid at each image acquisition moment to obtain the initial fused physical coordinates in each grid at each image acquisition moment; and The second fusion unit is used to fuse the initial fused physical coordinates in each network at the time of multiple image acquisition to obtain the fused physical coordinates.

15. A heat map generating device, comprising: A second sending module, used for sending the calibration coordinates for the scene view to the server; The calibration coordinates represent the size of the view area corresponding to the scene in the scene view; A first acquisition module, configured to acquire, in response to receiving the initial hotspot data from the server, a first height and a first width of the scene view at a first page resolution and an original height and an original width of the scene view at an original page resolution; A first conversion module, configured to convert the initial hotspot data into target hotspot data at a first page resolution based on the first height, the first width, the original height, and the original width; as well as The second generating module is used to generate a heat map within the scene based on the target hotspot data.

16. The device according to claim 15, wherein: The initial hotspot data includes initial fusion pixel coordinates and initial display radius; The first conversion module comprises: a fourth determining unit, configured to determine a first width ratio based on the first width and the original width; a fifth determining unit, configured to determine a first height ratio based on the first height and the original height; A second conversion unit, configured to convert the initial fused pixel coordinates into the target pixel coordinates according to the first width ratio and the first height ratio; A third conversion unit, configured to convert the initial display radius into a target display radius according to the first width ratio; and The sixth determination unit is used to determine the target hotspot data according to the target pixel coordinates and the target display radius.

17. The apparatus according to claim 15, further comprising: A second acquisition module, used to acquire a second height and a second width of the scene view at a second page resolution; A second receiving module, used to receive configuration information of the origin coordinates of the scene view; A calibration module is used to calibrate the view area in the scene view based on the origin coordinates of the scene view by using the second height and the second width as well as the original height and the original width to obtain calibration coordinates for the scene view.

18. The device according to claim 17, wherein: The calibration module comprises: A receiving unit, configured to receive a calibration operation for the viewing area to determine calibration position information for the viewing area; a seventh determining unit, configured to determine a second height ratio according to the second height and the original height; an eighth determining unit, configured to determine a second width ratio according to the second width and the original width; and A ninth determining unit is configured to determine the calibration coordinates based on the second height ratio, the second width ratio, and the calibration position information for the viewing area.

19. An image processing device, comprising: The third acquisition module is used to acquire the image collected at each image collection moment; A determination module, configured to determine pixel coordinates of at least one object in the image in an image coordinate system; A second conversion module, configured to convert pixel coordinates of the at least one object in the image coordinate system into second physical coordinates in the scene coordinate system relative to the origin coordinates of the image coordinate system; a correction module, configured to correct the second physical coordinates based on the correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, so as to obtain the first physical coordinates of the at least one object in the scene coordinate system; as well as The third sending module is used to send the first physical coordinates of the at least one object in the scene coordinate system to the server.

20. The device according to claim 19, wherein The correction module comprises: a tenth determining unit, configured to determine, based on a correspondence between the origin coordinates of the image coordinate system and the origin coordinates of the scene coordinate system, a coordinate correction value of the origin coordinates of the image coordinate system relative to the origin coordinates of the scene coordinate system in the scene coordinate system; and A correction unit is used to correct the second physical coordinate based on the coordinate correction value to obtain the first physical coordinate of the at least one object in the scene coordinate system.

21. An electronic device comprising a memory and a processor, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

24. A cloud server, comprising: The data processing device according to any one of claims 11 to 14.

25. A terminal, comprising: The thermal map generating device according to any one of claims 15 to 18.

26. An image acquisition device, comprising: The image processing device according to any one of claims 19 to 20.

Citation Information

Patent Citations

  • Map drawing method and device

    CN110956673A

  • Image imaging jitter compensation method, system and device

    CN111614895A

  • Parameter determination method, device, equipment and system

    CN111784781A

  • Image super-division method and electronic equipment

    CN114298900A

  • Moving target video positioning and speed measuring system and method

    CN115201883A